New handling of nested distribution groups in multiple distribution group migrations…
A question that is often asked is how to handle nested distribution list when considering a migration. When performing a single distribution list migration a recipient normalization process scans each member for attributes and object classes. If a group object class is found and is mail enabled the single group migration fails with an exception. The administrator must either first migrate the child distribution group or remove the mail enabled group. Here is a sample exception:
ParentGroupSMTPAddress : atestgroup27@domain.com
Alias : aTestGroup270
RecipientOrUser : Recipient
isError : True
isErrorMessage : NestedGroupException – A mail enabled group is a child member of the migrated list. The
child group must be migrated first or removed from group membership.
Name : aTestGroup270
GUID :
OnPremADAttribute : member
GroupType : -2147483640
isAlreadyMigrated : False
ExternalDirectoryObjectID :
DN : CN=aTestGroup270,OU=MigrationTest,OU=DLConversion,DC=home,DC=domain,DC=com
RecipientType :
OnPremADAttributeCommonName : Member
PrimarySMTPAddressOrUPN : aTestGroup270@domain.com
For customers that were evaluating options of migration distribution lists in bulk my original advice was to generate the list of primary SMTP addresses for migration. This list would then be run through the multiple migration process. In the end any group that could be migrated was migrated and any group left had an exception. The same set of addresses would be processed multiple times until no more migrations completed. The lists that were left required administrative intervention. The process overall was highly inefficient and sorting through the errors could be complicated. For some customers it was easier to iterate through the list rather than try to outline the dependencies and perform the migrations in order.
For customers that were able to map nested group membership the addresses would be sorted in correct order. The multiple migration process was called on the list and as long as there were no overlap in how the addresses were batched into groups of 5 the migrations would proceed successfully. Nested DLs were migrated first and the parent DLs later in the batch. If there was any overlap, for example a parent and child included in the same batch of simultaneous migrations, the parent would fail with the exception above.
For a review of how the module tracks nested membership and groups that have already been migrated see the following post. Office 365 – Distribution List Migrations Version 2.0 – Part 6 | TIMMCMIC (wordpress.com)
In version 2.9.5 new code is implemented in start-multipleDistributionListMigration that no longer requires the administrator to iterate through the list multiple times or pre-sort the list to account for nesting. The new code implements the following process:
Let examine a practical example of this using the following structure.
The array of SMTP addresses that the administrator specified for migration:
GroupA
GroupZ
GroupB
GroupC
GroupB1
GroupC1
GroupC2
GroupC3
GroupC4
GroupX
GroupY
The start-MultipleDistributionListMigration code breaks the groups down into the maximum number of threads a single host can migrate. The batch size is 5 groups per batch. The first batch of groups for migration includes:
GroupA
GroupZ
GroupB
GroupC
GroupB1
At the end of the first batch of migrations GroupB1 migrates successfully as it has no child group dependencies. GroupA, GroupZ, GroupB, and GroupC all fail with nested child groups. These groups are added to the retry list.
The next batch of groups includes:
GroupC1
GroupC2
GroupC3
GroupC4
GroupX
At the end of the second batch of migrations GroupC1, GroupC2, GroupC3, and GroupC4 all migrate successfully as they have no child group dependencies. GroupX fails with a nested child group exception and is added to the retry list.
The final batch is a single group – GroupY. GroupY will fail to migrate with a nested child group exception and will be added to the retry list.
At the conclusion of all first attempts the retry list contains the following groups:
GroupA
GroupZ
GroupB
GroupC
GroupX
GroupY
The groups are now scanned to determine if they are eligible for retry.
The first can searches the list for circular dependencies. A circular dependency is when two groups are members of each other. In this instance GroupX is a member of GroupY and GroupY is a member of GroupX. This is a cross dependency that cannot be reconciled. These groups hard fail and are added to the failure list for manual administrator intervention.
The second scan searches the list of exceptions. Each exception contains the ChildDL primary SMTP address and the ParentDL primary SMTP address. The list of original SMTP addresses is scanned and if the child groups primary SMTP address was present in the list to be migrated then the group is added for retry. Here is an example using our calculated retry list.
GroupA@domain.com -> GroupB@domain.com (GroupB@domain.com was originally included add GroupA to the retry list).
GroupA@domain.com -> GroupC@domain.com (GroupC@domain.com was originally included add GroupA to the retry list).
GroupZ@domain.com -> GroupZ1@domain.com (GroupZ1@domain.com was NOT originally included. GroupZ will fail and be added to the failure list for manual administrator intervention).
GroupB@domain.com -> GroupB1@domain.com (GroupB1@domain.com was originally included add GroupB to the retry list).
GroupC@domain.com -> GroupC1@domain.com (GroupC1@domain.com was originally included add GroupC to the retry list).
GroupC@domain.com -> GroupC2@domain.com (GroupC2@domain.com was originally included add GroupC to the retry list).
GroupC@domain.com -> GroupC3@domain.com (GroupC3@domain.com was originally included add GroupC to the retry list).
GroupC@domain.com -> GroupC4@domain.com (GroupC4@domain.com was originally included add GroupC to the retry list).
At the end of child validation scanning the list is as follows:
GroupA
GroupA
GroupB
GroupC
GroupC
GroupC
GroupC
The results are trimmed into a unique list to retry:
GroupA
GroupB
GroupC
The code then retries the migration using the updated list of addresses to retry. The list contains less than 5 members only a single batch is created. At the conclusion of this second attempt GroupB and GroupC migrate successfully as there are no nested child lists in the group any longer. GroupA fails since at the time of the retry GroupB and GroupC were both members. The process now repeats itself using the new nested exception list.
GroupA@domain.com -> GroupB@domain.com (GroupB@domain.com was originally included add GroupA to the retry list).
GroupA@domain.com -> GroupC@domain.com (GroupC@domain.com was originally included add GroupA to the retry list).
This results in the following retry list:
GroupA
GroupA
The results are then trimmed into a unique list to retry.
The code then retries the migration using the updated addresses to retry. The list in this case contains only one address so a single batch is created. At the conclusion GroupA now migrates successfully as there are no nested child group dependencies.
The migration is concluded at this time.
GroupZ will not migrate due to a child group not included in the migration set. Here is an example of an exception for this reason:
[1/6/2023 12:20:56 AM] – =====
[1/6/2023 12:20:56 AM] – Alias: a300
[1/6/2023 12:20:56 AM] – Name: a300
[1/6/2023 12:20:56 AM] – PrimarySMTPAddressOrUPN: a300@domain.com
[1/6/2023 12:20:56 AM] – RecipientType: group
[1/6/2023 12:20:56 AM] – GroupType: -2147483640
[1/6/2023 12:20:56 AM] – RecipientOrUser: Recipient
[1/6/2023 12:20:56 AM] – ExternalDirectoryObjectID:
[1/6/2023 12:20:56 AM] – OnPremADAttribute: member
[1/6/2023 12:20:56 AM] – DN: CN=a300,OU=DoNotSync,OU=MigrationTest,OU=DLConversion,DC=home,DC=domain,DC=com
[1/6/2023 12:20:56 AM] – ParentGroupSMTPAddress: aTopZ@domain.com
[1/6/2023 12:20:56 AM] – isAlreadyMigrated: False
[1/6/2023 12:20:56 AM] – isError: True
[1/6/2023 12:20:56 AM] – isErrorMessage: ChildGroupMirationException: The groupt to be migrated has a child group not included in the migration set.
[1/6/2023 12:20:56 AM] – =====
GroupX and GroupY will fail to migrate due to a circular dependency. Here is an example of an exception for this reason:
[1/6/2023 12:20:56 AM] – =====
[1/6/2023 12:20:56 AM] – Alias: aTopY
[1/6/2023 12:20:56 AM] – Name: aTopY
[1/6/2023 12:20:56 AM] – PrimarySMTPAddressOrUPN: aTopY@domain.com
[1/6/2023 12:20:56 AM] – RecipientType: group
[1/6/2023 12:20:56 AM] – GroupType: -2147483640
[1/6/2023 12:20:56 AM] – RecipientOrUser: Recipient
[1/6/2023 12:20:56 AM] – ExternalDirectoryObjectID:
[1/6/2023 12:20:56 AM] – OnPremADAttribute: member
[1/6/2023 12:20:56 AM] – DN: CN=aTopY,OU=RecursionTest,OU=MigrationTest,OU=DLConversion,DC=home,DC=domain,DC=com
[1/6/2023 12:20:56 AM] – ParentGroupSMTPAddress: aTopX@domain.com
[1/6/2023 12:20:56 AM] – isAlreadyMigrated: False
[1/6/2023 12:20:56 AM] – isError: True
[1/6/2023 12:20:56 AM] – isErrorMessage: CircularReferenceException: This group has a child distribution list that also has this group as a member. This creates a circular dependency which cannot be handeled automatically.
[1/6/2023 12:20:56 AM] – =====
What happens if a group fails to migrate for another reason? If a child group fails to migrate due to another error besides a nested group exception the group is not written to the CSV file. This means that when the pass is completed and the CSV file is analyzed – the child DL will no longer be found as included in the migration set. This yields an error that the child group was not present in the migration set. It is quite confusing because it was included in the migration set but unfortunately errored for another reason.
I hope this change allows mass migrations to proceed with greater success and less administrator intervention. Happy migrating!
TIMMCMIC 