Increasing the performance of multiple distribution list migrations.

In the following blog post I outlined how the multiple distribution list migration process works.

Office 365 – Distribution List Migrations Version 2.0 – Part 9 | TIMMCMIC (wordpress.com)

In a multiple migration scenario each distribution list in a batch is assigned to a PowerShell job that performs a single distribution list migration. There are certain operations that it is desirable to only perform one time across the migration batch of five groups the main one being Azure Active Directory Connect delta synchronization. The individual distribution list migrations are designed so that all migrations reach a certain point before triggering the operations that we only want performed one time. The first migration thread one is responsible for triggering these operations once threads two through five are ready.

There could be any number of issues that cause the distribution list assigned to thread one to fail migration. For example, if a distribution list member is not present in Office 365 the migration of the distribution list will fail as a perquisite exists that all members exist in Office 365 before performing a migration. Originally when thread one failed the performance of the other distribution list migrations were negatively impacted specifically by the lack of triggering of Azure Active Directory Connect delta synchronization. In Azure Active Directory Connect the default sync cycle runs every 30 minutes. If thread one fails and cannot trigger the delta synchronization it could be up to 30 minutes before it would automatically run. This would add 30 minutes to the overall migration time.

In version 2.9.5 thread two through five now track the lack of removal of the distribution group from Office 365. If the distribution group is not removed from Office 365 in 5 minutes thread two through five will now attempt a delta synchronization in Azure Active Directory Connect. This cycle repeats itself for every 5 minutes that expire waiting for deletion. It is important to note that there could be service side sync delays which contribute to the amount of time waiting for the distribution list to be deleted from Exchange Online. The assumption is made that if the group is there after 5 minutes it was due to a thread one failure and Azure Active Directory Connect should be triggered.

Does this lead to multiple delta sync cycles being run by multiple threads? Generally, not – since each thread has reached the same point in the migration process prior to monitoring Office 365 for the group deletion they will all attempt to trigger Azure Active Directory Connect around the same time. Only one synchronization cycle can proceed at a time therefore one thread will receive a success message that synchronization was triggered, and other threads will receive an error that synchronization is busy. Threads two through 5 do not retry their sync operations but rather simple call the sync process one time. Thread one does have retry logic and always enforces that Azure Active Directory Connect delta sync was triggered one time in the migration process.

By assuming that there is a thread one failure when the distribution list deletion does not occur in a timely fashion and triggering Azure Active Directory Connect we can eliminate undesirable delays in multiple distribution list migrations.