In Exchange 2010 administrators may attempt to move either log files or database files using the command move-DatabasePath.  In some instances the command may fail containing an InvalidOperationException.  Here are some examples of the errors (note that verbose output is utilized):

========================================================

Move-DatabasePath –identity <name> –logFolderPath <path> –verbose

[PS] C:>Move-DatabasePath MBX-1-DB0 -LogFolderPath T:NewPath -Verbose
VERBOSE: [17:29:43.342 GMT] Move-DatabasePath : Active Directory session
settings for ‘Move-DatabasePath’ are: View Entire Forest: ‘False’, Default
Scope: ‘home.domain.com’, Configuration Domain Controller:
‘DC-1.home.domain.com’, Preferred Global Catalog:
‘DC-2.home.domain.com’, Preferred Domain Controllers: ‘{
DC-2.home.domain.com }’
VERBOSE: [17:29:43.342 GMT] Move-DatabasePath : Runspace context: Executing
user: home.domain.com/Users/Administrator, Executing user organization: ,
Current organization: , RBAC-enabled: Enabled.
VERBOSE: [17:29:43.342 GMT] Move-DatabasePath : Beginning processing &
VERBOSE: [17:29:43.342 GMT] Move-DatabasePath : Instantiating handler with
index 0 for cmdlet extension agent "Admin Audit Log Agent".
VERBOSE: [17:29:43.420 GMT] Move-DatabasePath : Current ScopeSet is: {
Recipient Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration
Read Scope: {{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient
Scope(s): {}, Exclusive Configuration Scope(s): {} }
VERBOSE: [17:29:44.124 GMT] Move-DatabasePath : Searching objects "MBX-1-DB0"
of type "Database" under the root "$null".
VERBOSE: [17:29:44.155 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [17:29:44.217 GMT] Move-DatabasePath : Processing object "MBX-1-DB0".
VERBOSE: [17:29:45.186 GMT] Move-DatabasePath : Searching objects
"MBX-1.home.domain.com" of type "Server" under the root "$null".
VERBOSE: [17:29:45.280 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [17:29:46.873 GMT] Move-DatabasePath : Verifying that the path
"T:NewPath" on the server "MBX-1.home.domain.com" is located on a fixed
or network drive.
VERBOSE: [17:29:49.529 GMT] Move-DatabasePath : Verifying that the log location
"T:NewPath" on server "MBX-1.home.domain.com" has not been occupied by
an existing file or directory.
VERBOSE: [17:29:50.326 GMT] Move-DatabasePath : Checking the existence of log
files under directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" on Server
"MBX-1.home.domain.com".
VERBOSE: [17:33:05.815 GMT] Move-DatabasePath : Admin Audit Log: Entered
Handler:Validate.
VERBOSE: [17:33:05.815 GMT] Move-DatabasePath : Admin Audit Log: Entered
ClassFactory:InitializeConfig.
VERBOSE: [17:33:05.924 GMT] Move-DatabasePath : Admin Audit Log: Exited
ClassFactory:InitializeConfig.
VERBOSE: [17:33:07.190 GMT] Move-DatabasePath : Admin Audit Log: Exited
Handler:Validate.

Confirm
Are you sure you want to perform this action?

Moving database path "MBX-1-DB0".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [17:33:52.375 GMT] Move-DatabasePath : Resolved current organization:
.
VERBOSE: [17:33:52.594 GMT] Move-DatabasePath : The Admin RPC connection is
being established with server "MBX-1.home.domain.com".

Confirm
To perform the move operation, database "MBX-1-DB0" must be temporarily
dismounted, which will make it inaccessible to all users. Do you want to
continue?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [17:34:04.422 GMT] Move-DatabasePath : Reading new object
"9d654028-b845-43fc-9b3b-a011c442c9e6" of type "Server".
VERBOSE: [17:34:04.422 GMT] Move-DatabasePath : Dismounting the database,
"MBX-1-DB0".
VERBOSE: [17:34:05.297 GMT] Move-DatabasePath : Copying the database Log files
under directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" to the target directory
"T:NewPath" on Server "MBX-1".
VERBOSE: [17:34:05.344 GMT] Move-DatabasePath : Checking the existence of
directory "T:NewPath" on Server "MBX-1.home.domain.com".
VERBOSE: [17:34:05.359 GMT] Move-DatabasePath : Set access control for
directory "T:NewPath" on server "MBX-1.home.domain.com".
VERBOSE: [17:34:05.359 GMT] Move-DatabasePath : Checking the existence of
directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" on Server
"MBX-1.home.domain.com".
VERBOSE: [17:34:05.406 GMT] Move-DatabasePath : Copying the files from
directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" to target directory "T:NewPath"
on server "MBX-1.home.domain.com".
VERBOSE: [17:35:54.090 GMT] Move-DatabasePath : Reading new object
"9d654028-b845-43fc-9b3b-a011c442c9e6" of type "Server".
VERBOSE: [17:35:54.105 GMT] Move-DatabasePath : The task is attempting to
communicate with the Microsoft Exchange Replication service on server
"MBX-1.home.domain.com" to obtain updated configuration changes for
database "MBX-1-DB0".
VERBOSE: [17:35:54.965 GMT] Move-DatabasePath : Mounting database "MBX-1-DB0".
VERBOSE: [17:35:56.199 GMT] Move-DatabasePath : Admin Audit Log: Entered
Handler:OnComplete.
VERBOSE: [17:36:03.902 GMT] Move-DatabasePath : Admin Audit Log: Exited
Handler:OnComplete.
Failed to move database log files from "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" to "
T:NewPath".
    + CategoryInfo          : InvalidOperation: (MBX-1-DB0:ADObjectId) [Move-DatabasePath], InvalidOperationException
    + FullyQualifiedErrorId : 7280D023,Microsoft.Exchange.Management.SystemConfigurationTasks.MoveDatabasePath
VERBOSE: [17:36:04.074 GMT] Move-DatabasePath : Ending processing &
[PS] C:>

========================================================

Move-DatabasePath –identity <name> –logFolderPath <path> –edbFilePath <pathname.edb> –verbose

[PS] C:>Move-DatabasePath -Identity MBX-1-DB0 -EdbFilePath t:NewPathMBX-1-DB0.edb

Confirm
Are you sure you want to perform this action?

Moving database path "MBX-1-DB0".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a

Confirm
To perform the move operation, database "MBX-1-DB0" must be temporarily
dismounted, which will make it inaccessible to all users. Do you want to
continue?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
[PS] C:>Move-DatabasePath -Identity MBX-1-DB0 -EdbFilePath "E:MBX-1MBX-1-DB0MBX-1-DB0-DatabaseMBX-1-DB0.edb"

Confirm
Are you sure you want to perform this action?

Moving database path "MBX-1-DB0".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a

Confirm
To perform the move operation, database "MBX-1-DB0" must be temporarily
dismounted, which will make it inaccessible to all users. Do you want to
continue?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
[PS] C:>       Move-DatabasePath MBX-1-DB0 -LogFolderPath T:NewPath -EdbFilePath t:NewPathMBX-1-DB0.edb -Verbose
VERBOSE: [17:43:42.454 GMT] Move-DatabasePath : Active Directory session
settings for ‘Move-DatabasePath’ are: View Entire Forest: ‘False’, Default
Scope: ‘home.domain.com’, Configuration Domain Controller:
‘DC-1.home.domain.com’, Preferred Global Catalog:
‘DC-2.home.domain.com’, Preferred Domain Controllers: ‘{
DC-2.home.domain.com }’
VERBOSE: [17:43:42.454 GMT] Move-DatabasePath : Runspace context: Executing
user: home.domain.com/Users/Administrator, Executing user organization: ,
Current organization: , RBAC-enabled: Enabled.
VERBOSE: [17:43:42.454 GMT] Move-DatabasePath : Beginning processing &
VERBOSE: [17:43:42.454 GMT] Move-DatabasePath : Instantiating handler with
index 0 for cmdlet extension agent "Admin Audit Log Agent".
VERBOSE: [17:43:42.470 GMT] Move-DatabasePath : Current ScopeSet is: {
Recipient Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration
Read Scope: {{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient
Scope(s): {}, Exclusive Configuration Scope(s): {} }
VERBOSE: [17:43:42.470 GMT] Move-DatabasePath : Searching objects "MBX-1-DB0"
of type "Database" under the root "$null".
VERBOSE: [17:43:42.486 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [17:43:42.486 GMT] Move-DatabasePath : Processing object "MBX-1-DB0".
VERBOSE: [17:43:42.501 GMT] Move-DatabasePath : Searching objects
"MBX-1.home.domain.com" of type "Server" under the root "$null".
VERBOSE: [17:43:42.533 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [17:43:42.595 GMT] Move-DatabasePath : Verifying that EDB file path
"t:NewPathMBX-1-DB0.edb" is available for use.
VERBOSE: [17:43:42.704 GMT] Move-DatabasePath : Checking the existence of file
"E:MBX-1MBX-1-DB0MBX-1-DB0-DatabaseMBX-1-DB0.edb" on server "MBX-1".
VERBOSE: [17:43:42.783 GMT] Move-DatabasePath : Verifying that the path
"t:NewPathMBX-1-DB0.edb" on the server "MBX-1" is located on a fixed or
network drive.
VERBOSE: [17:43:42.783 GMT] Move-DatabasePath : Checking the existence of file
"t:NewPathMBX-1-DB0.edb" on server "MBX-1".
VERBOSE: [17:43:42.814 GMT] Move-DatabasePath : Checking the existence of
directory "t:NewPathMBX-1-DB0.edb" on Server "MBX-1".
VERBOSE: [17:43:42.845 GMT] Move-DatabasePath : Checking the existence of
directory "t:NewPath" on Server "MBX-1.home.domain.com".
VERBOSE: [17:43:42.845 GMT] Move-DatabasePath : Set access control for
directory "t:NewPath" on server "MBX-1.home.domain.com".
VERBOSE: [17:43:42.845 GMT] Move-DatabasePath : Checking the existence of
directory
"e:mbx-1mbx-1-db0mbx-1-db0-databasecatalogdata-ed71935d-a14f-4937-b20d-50b3
7e136797-9d654028-b845-43fc-9b3b-a011c442c9e6" on Server "MBX-1".
VERBOSE: [17:43:42.845 GMT] Move-DatabasePath : Verifying that the path
"T:NewPath" on the server "MBX-1.home.domain.com" is located on a fixed
or network drive.
VERBOSE: [17:43:42.861 GMT] Move-DatabasePath : Verifying that the log location
"T:NewPath" on server "MBX-1.home.domain.com" has not been occupied by
an existing file or directory.
VERBOSE: [17:43:42.923 GMT] Move-DatabasePath : Checking the existence of log
files under directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" on Server
"MBX-1.home.domain.com".
VERBOSE: [17:46:51.018 GMT] Move-DatabasePath : Admin Audit Log: Entered
Handler:Validate.
VERBOSE: [17:46:51.034 GMT] Move-DatabasePath : Admin Audit Log: Exited
Handler:Validate.

Confirm
Are you sure you want to perform this action?

Moving database path "MBX-1-DB0".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [17:51:35.320 GMT] Move-DatabasePath : Resolved current organization:
.
VERBOSE: [17:51:35.351 GMT] Move-DatabasePath : The Admin RPC connection is
being established with server "MBX-1.home.domain.com".

Confirm
To perform the move operation, database "MBX-1-DB0" must be temporarily
dismounted, which will make it inaccessible to all users. Do you want to
continue?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [17:51:37.586 GMT] Move-DatabasePath : Reading new object
"9d654028-b845-43fc-9b3b-a011c442c9e6" of type "Server".
VERBOSE: [17:51:37.586 GMT] Move-DatabasePath : Dismounting the database,
"MBX-1-DB0".
VERBOSE: [17:51:37.851 GMT] Move-DatabasePath : The file is being copied from
"E:MBX-1MBX-1-DB0MBX-1-DB0-DatabaseMBX-1-DB0.edb" to
"t:NewPathMBX-1-DB0.edb" on server "MBX-1.home.domain.com".
VERBOSE: [17:53:03.177 GMT] Move-DatabasePath : Copying the database Log files
under directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" to the target directory
"T:NewPath" on Server "MBX-1".
VERBOSE: [17:53:03.209 GMT] Move-DatabasePath : Checking the existence of
directory "T:NewPath" on Server "MBX-1.home.domain.com".
VERBOSE: [17:53:03.209 GMT] Move-DatabasePath : Set access control for
directory "T:NewPath" on server "MBX-1.home.domain.com".
VERBOSE: [17:53:03.209 GMT] Move-DatabasePath : Checking the existence of
directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" on Server
"MBX-1.home.domain.com".
VERBOSE: [17:53:03.271 GMT] Move-DatabasePath : Copying the files from
directory "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" to target directory "T:NewPath"
on server "MBX-1.home.domain.com".
VERBOSE: [17:56:19.237 GMT] Move-DatabasePath : Deleting the file
"t:NewPathMBX-1-DB0.edb" on server "MBX-1".
VERBOSE: [17:56:19.378 GMT] Move-DatabasePath : Reading new object
"9d654028-b845-43fc-9b3b-a011c442c9e6" of type "Server".
VERBOSE: [17:56:19.378 GMT] Move-DatabasePath : The task is attempting to
communicate with the Microsoft Exchange Replication service on server
"MBX-1.home.domain.com" to obtain updated configuration changes for
database "MBX-1-DB0".
VERBOSE: [17:56:19.503 GMT] Move-DatabasePath : Mounting database "MBX-1-DB0".
VERBOSE: [17:56:21.487 GMT] Move-DatabasePath : Admin Audit Log: Entered
Handler:OnComplete.
VERBOSE: [17:56:21.831 GMT] Move-DatabasePath : Admin Audit Log: Exited
Handler:OnComplete.
Failed to move database log files from "d:MBX-1MBX-1-DB0MBX-1-DB0-Logs" to "
T:NewPath".
    + CategoryInfo          : InvalidOperation: (MBX-1-DB0:ADObjectId) [Move-DatabasePath], InvalidOperationException
    + FullyQualifiedErrorId : 7280D023,Microsoft.Exchange.Management.SystemConfigurationTasks.MoveDatabasePath
VERBOSE: [17:56:22.003 GMT] Move-DatabasePath : Ending processing &
[PS] C:>

========================================================

The issue that causes this error is due to the way free space is determined when a mount point is utilized.  In this case we have the following disk layout:

Disk T: –> 8 megs free space.

Disk T:NewPath –>  NewPath is a mount point created from a folder residing on the T volume.  NewPath has 127 gig of free space.

In this case the first check that is performed is a WMI query to determine if the destination has enough free space to accommodate the log files being moved.  Unfortunately the WMI call utilized checks the free space of the mount point root disk, and not the mount point itself.  This results in the destination being preceived as not having enough disk space and the move operation returning failed with an exception.

Note:  This issue only happens when moving log files or moving log files with a database.  If only the database path is adjusted this issue does not occur.

To correct this issue the log files can be moved by hand.  The following steps will work around this condition:

1)  Dismount the database that you want to move log files for.

Dismount-Database –identity <NAME> -verbose

[PS] C:>Dismount-Database -Identity MBX-1-DB0 -Verbose
VERBOSE: [18:12:18.536 GMT] Dismount-Database : Active Directory session
settings for ‘Dismount-Database’ are: View Entire Forest: ‘False’, Default
Scope: ‘home.domain.com’, Configuration Domain Controller:
‘DC-1.home.domain.com’, Preferred Global Catalog:
‘DC-2.home.domain.com’, Preferred Domain Controllers: ‘{
DC-2.home.domain.com }’
VERBOSE: [18:12:18.536 GMT] Dismount-Database : Runspace context: Executing
user: home.domain.com/Users/Administrator, Executing user organization: ,
Current organization: , RBAC-enabled: Enabled.
VERBOSE: [18:12:18.536 GMT] Dismount-Database : Beginning processing &
VERBOSE: [18:12:18.536 GMT] Dismount-Database : Instantiating handler with
index 0 for cmdlet extension agent "Admin Audit Log Agent".
VERBOSE: [18:12:18.583 GMT] Dismount-Database : Current ScopeSet is: {
Recipient Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration
Read Scope: {{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient
Scope(s): {}, Exclusive Configuration Scope(s): {} }
VERBOSE: [18:12:18.583 GMT] Dismount-Database : Searching objects "MBX-1-DB0"
of type "Database" under the root "$null".
VERBOSE: [18:12:18.598 GMT] Dismount-Database : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [18:12:18.598 GMT] Dismount-Database : Processing object "MBX-1-DB0".
VERBOSE: [18:12:18.645 GMT] Dismount-Database : Admin Audit Log: Entered
Handler:Validate.
VERBOSE: [18:12:18.645 GMT] Dismount-Database : Admin Audit Log: Exited
Handler:Validate.

Confirm
Are you sure you want to perform this action?

Dismounting database "MBX-1-DB0". This may result in reduced availability for
mailboxes in the database.
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [18:12:20.458 GMT] Dismount-Database : Resolved current organization:
.
VERBOSE: [18:12:20.489 GMT] Dismount-Database : The Admin RPC connection is
being established with server "MBX-1.home.domain.com".
VERBOSE: [18:12:20.489 GMT] Dismount-Database : Dismounting the database,
"MBX-1-DB0".
VERBOSE: [18:12:20.817 GMT] Dismount-Database : Admin Audit Log: Entered
Handler:OnComplete.
VERBOSE: [18:12:20.880 GMT] Dismount-Database : Admin Audit Log: Exited
Handler:OnComplete.
VERBOSE: [18:12:20.895 GMT] Dismount-Database : Ending processing &
[PS] C:>

2)  Move the log file path using the configuration only switch which will simply write the new path to the active directory. 

Move-DatabasePath –identity <NAME> –logFolderPath <path> –configurationOnly:$TRUE –verbose

[PS] C:>Move-DatabasePath -Identity MBX-1-DB0 -LogFolderPath T:NewPath -ConfigurationOnly:$TRUE -Verbose
VERBOSE: [18:18:12.047 GMT] Move-DatabasePath : Active Directory session
settings for ‘Move-DatabasePath’ are: View Entire Forest: ‘False’, Default
Scope: ‘home.domain.com’, Configuration Domain Controller:
‘DC-1.home.domain.com’, Preferred Global Catalog:
‘DC-2.home.domain.com’, Preferred Domain Controllers: ‘{
DC-2.home.domain.com }’
VERBOSE: [18:18:12.063 GMT] Move-DatabasePath : Runspace context: Executing
user: home.domain.com/Users/Administrator, Executing user organization: ,
Current organization: , RBAC-enabled: Enabled.
VERBOSE: [18:18:12.063 GMT] Move-DatabasePath : Beginning processing &
VERBOSE: [18:18:12.063 GMT] Move-DatabasePath : Instantiating handler with
index 0 for cmdlet extension agent "Admin Audit Log Agent".
VERBOSE: [18:18:12.063 GMT] Move-DatabasePath : Current ScopeSet is: {
Recipient Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration
Read Scope: {{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient
Scope(s): {}, Exclusive Configuration Scope(s): {} }
VERBOSE: [18:18:12.079 GMT] Move-DatabasePath : Searching objects "MBX-1-DB0"
of type "Database" under the root "$null".
VERBOSE: [18:18:12.360 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [18:18:12.360 GMT] Move-DatabasePath : Processing object "MBX-1-DB0".
VERBOSE: [18:18:12.391 GMT] Move-DatabasePath : Searching objects
"MBX-1.home.domain.com" of type "Server" under the root "$null".
VERBOSE: [18:18:12.391 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.

Confirm
This operation will skip the safety check and make the change to Active
Directory directly. Do you want to continue?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [18:18:14.798 GMT] Move-DatabasePath : Admin Audit Log: Entered
Handler:Validate.
VERBOSE: [18:18:14.798 GMT] Move-DatabasePath : Admin Audit Log: Exited
Handler:Validate.

Confirm
Are you sure you want to perform this action?

Moving database path "MBX-1-DB0".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): a
VERBOSE: [18:18:16.110 GMT] Move-DatabasePath : Resolved current organization:
.
VERBOSE: [18:18:16.126 GMT] Move-DatabasePath : The Admin RPC connection is
being established with server "MBX-1.home.domain.com".
VERBOSE: [18:18:16.126 GMT] Move-DatabasePath : Saving object "MBX-1-DB0" of
type "Database" and state "Changed".
VERBOSE: [18:18:16.126 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [18:18:16.126 GMT] Move-DatabasePath : Getting the name of the domain
controller to be used to double write the configurable object’s changes. Double
write is the mechanism to make sure that the changes are persisted in the
domain controller used by the store service on server
"MBX-1.home.domain.com".
VERBOSE: [18:18:16.141 GMT] Move-DatabasePath : Reading new object "MBX-1-DB0"
of type "Database".
VERBOSE: [18:18:16.141 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [18:18:16.141 GMT] Move-DatabasePath : Double writing changes of the
configurable object, "MBX-1-DB0", on domain controller
"DC-1.home.domain.com".
VERBOSE: [18:18:16.141 GMT] Move-DatabasePath : Previous operation run on
domain controller ‘DC-1.home.domain.com’.
VERBOSE: [18:18:16.141 GMT] Move-DatabasePath : The task is attempting to
communicate with the Microsoft Exchange Replication service on server
"MBX-1.home.domain.com" to obtain updated configuration changes for
database "MBX-1-DB0".
VERBOSE: [18:18:16.298 GMT] Move-DatabasePath : Admin Audit Log: Entered
Handler:OnComplete.
VERBOSE: [18:18:16.313 GMT] Move-DatabasePath : Admin Audit Log: Exited
Handler:OnComplete.
VERBOSE: [18:18:16.313 GMT] Move-DatabasePath : Ending processing &
[PS] C:>

3)  Using ROBOCopy – which is included with Windows 2008 and Windows 2008 R2 – mirror the current log directory to the new location. 

Note:  This command will MOVE the files – that is copy them to the new directory and then delete the original file.

Note:  This command assumes that only log files exist in the directory.  If the database and catalog files exist in the directory they should be skipped.

ROBOCOPY <SOURCE-DIR> <TARGET-DIR> /MOVE /XD *Catalog* /XF *.edb

[PS] C:>robocopy "D:MBX-1MBX-1-DB0MBX-1-DB0-Logs" "T:NewPath" /MOVE /XD *Catalog* /XF *.edb

——————————————————————————-
   ROBOCOPY     ::     Robust File Copy for Windows

——————————————————————————-

  Started : Mon Sep 27 14:50:08 2010

   Source : D:MBX-1MBX-1-DB0MBX-1-DB0-Logs
     Dest : T:NewPath

    Files : *.*

Exc Files : *.edb

Exc Dirs : *Catalog*

  Options : *.* /COPY:DAT /MOVE /R:1000000 /W:30

——————————————————————————

100%        New File               1.0 m        E00res00005.jrs
100%        New File               1.0 m        E00res00006.jrs
100%        New File               1.0 m        E00res00007.jrs
100%        New File               1.0 m        E00res00008.jrs
100%        New File               1.0 m        E00res00009.jrs
100%        New File               1.0 m        E00res0000A.jrs
100%        New File                   0        E00tmp.log

——————————————————————————

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         2         0         2         0         0         1
   Files :      6430      6429         1         0         0         0
   Bytes :   9.158 g   6.275 g   2.882 g         0         0         0
   Times :   0:04:03   0:03:40                       0:00:00   0:00:22

   Speed :            30539839 Bytes/sec.
   Speed :            1747.503 MegaBytes/min.

   Ended : Mon Sep 27 14:44:39 2010
[PS] C:>

3)  Optional:  Move the database file to the new location.

Move-DatabasePath –identity <NAME> –edbFilePath <TargetPathName.edb> –verbose

4)  Mount the database post all move operations (allow sufficient time for AD replication to occur)

Mount-Database –identity <NAME> –verbose

[PS] C:>Mount-Database -Identity MBX-1-DB0 -Verbose
VERBOSE: [19:06:06.471 GMT] Mount-Database : Active Directory session settings
for ‘Mount-Database’ are: View Entire Forest: ‘False’, Default Scope:
‘home.domain.com’, Configuration Domain Controller:
‘DC-1.home.domain.com’, Preferred Global Catalog:
‘DC-2.home.domain.com’, Preferred Domain Controllers: ‘{
DC-2.home.domain.com }’
VERBOSE: [19:06:06.486 GMT] Mount-Database : Runspace context: Executing user:
home.domain.com/Users/Administrator, Executing user organization: ,
Current organization: , RBAC-enabled: Enabled.
VERBOSE: [19:06:06.486 GMT] Mount-Database : Beginning processing &
VERBOSE: [19:06:06.486 GMT] Mount-Database : Instantiating handler with index 0
for cmdlet extension agent "Admin Audit Log Agent".
VERBOSE: [19:06:07.033 GMT] Mount-Database : Current ScopeSet is: { Recipient
Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration Read Scope:
{{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient Scope(s):
{}, Exclusive Configuration Scope(s): {} }
VERBOSE: [19:06:07.033 GMT] Mount-Database : Searching objects "MBX-1-DB0" of
type "Database" under the root "$null".
VERBOSE: [19:06:07.049 GMT] Mount-Database : Previous operation run on domain
controller ‘DC-1.home.domain.com’.
VERBOSE: [19:06:07.065 GMT] Mount-Database : Processing object "MBX-1-DB0".
VERBOSE: [19:06:07.111 GMT] Mount-Database : Admin Audit Log: Entered
Handler:Validate.
VERBOSE: [19:06:07.111 GMT] Mount-Database : Admin Audit Log: Exited
Handler:Validate.
VERBOSE: Mounting database "MBX-1-DB0".
VERBOSE: [19:06:07.127 GMT] Mount-Database : Resolved current organization: .
VERBOSE: [19:06:07.158 GMT] Mount-Database : Searching objects
"home.domain.com/Microsoft Exchange System
Objects/SystemMailbox{ed71935d-a14f-4937-b20d-50b37e136797}" of type
"ADRecipient" under the root "$null".
VERBOSE: [19:06:07.752 GMT] Mount-Database : Mounting database "MBX-1-DB0".
VERBOSE: [19:06:08.971 GMT] Mount-Database : Admin Audit Log: Entered
Handler:OnComplete.
VERBOSE: [19:06:09.065 GMT] Mount-Database : Admin Audit Log: Exited
Handler:OnComplete.
VERBOSE: [19:06:09.065 GMT] Mount-Database : Ending processing &
[PS] C:>

At this time we expect this issue to be resolved in Exchange 2010 Service Pack 2.

Recently some customers have experienced an issue where database copies do not display when using the Exchange Management Console after upgrading to Exchange 2010 Service Pack 1.  When attempting to view database copies using the Exchange Management Shell no issue is displayed. 

Database copies can be viewed in two locations. 

The first location is under Organization Configuration –> Mailbox –> Database Management.  In this view when an administrator selects a database from the list, the Database Copies displayed in the bottom portion of the display are missing ALL or SOME copies for a given database.  Here is an example of a 4 node DAG with a database replicated to all 4 members:

All Database Copies Missing:

Some Database Copies Missing:

Expected output showing all database copies:

The second location is under Server Configuration –> Mailbox.  By selecting a server with the mailbox role you can view the individual database copies assigned to that server.  Here is an example of database copies missing from the server:

Whether all database copies are missing <or> some database copies are missing the Exchange Management Shell command Get-MailboxDatabaseCopyStatus always returns accurate information.  Here is a copy of Get-MailboxDatabaseCopyStatus *:

When any Exchange role is installed on the server (with exception of Edge transport) the hostname of the server is written into the Exchange configuration container within Active Directory.  In this case the server name was established with a lower case.  It was established in the container with a lower case because the hostname of the server, which was established during setup, was established with a lower case name (or a portion of the name lowercase).

In reference customers running HOSTNAME <or> reviewing the full computer name in server management showed that all or a portion of the name was lower case.

The Exchange Management Console code incorrectly compares case when comparing a database copy against a server name.  This causes the console to not display all the valid database copies.

Let’s step through an example.

In this example there is a four node DAG.  The server names are as follows:

Dag-1

dag-2

DAG-3

DAG-4

You can verify these server names by running get-databaseavailabilitygroup –identity <DAGName> | fl name,servers

[PS] D:>Get-DatabaseAvailabilityGroup -Identity DAG | fl name,servers

Name    : DAG
Servers : {DAG-4, DAG-3, dag-2, Dag-1}

When looking in the management console, and selecting any database replicated to all four members, the only copies that will be displayed are on nodes DAG-4 and DAG-3.

When the Exchange Management Console draws the database copies pane, it compares the host server name of a database copy to the server name of a database copy status.  This comparison is case sensitive.  Let’s take a look at a database copy that fails to display when viewing the copies for DAG-1.

(Get-MailboxDatabase –identity <NAME>).databasecopies | fl hostservername

[PS] D:>(Get-MailboxDatabase DAG-DB0).databasecopies | fl hostservername

HostServerName : Dag-1

HostServerName : DAG-3

HostServerName : DAG-4

HostServerName : dag-2

(Get-MailboxDatabaseCopyStatus <NAME><Server>).mailboxserver

[PS] D:>(Get-MailboxDatabaseCopyStatus DAG-DB0DAG-1).mailboxServer
DAG-1

In this case DAG-1 != Dag-1 and therefore the copy does not display in the management console.

Let’s take a look at a copy that does display.

(Get-MailboxDatabase –identity <NAME>).databasecopies | fl hostservername

[PS] D:>(Get-MailboxDatabase DAG-DB0).databasecopies | fl hostservername

HostServerName : Dag-1

HostServerName : DAG-3

HostServerName : DAG-4

HostServerName : dag-2

(Get-MailboxDatabaseCopyStatus <NAME><Server>).mailboxserver

[PS] D:>(Get-MailboxDatabaseCopyStatus DAG-DB0DAG-4).mailboxServer
DAG-4

In this case DAG-4 == DAG-4 and therefore the copy does display in the management console.

Unfortunately at this time we can only recommend that database copies for missing servers be managed using the Exchange Management Shell.  It is not recommended to attempt to modify the active directory at this time to overcome this issue.

At this time we are filing the necessary bugs to get this permanently corrected without modifying server names.  I will update this blog as this process progresses.

=======================================

Update 10/10/2010

This issue is currently scheduled to be corrected in Exchange 2010 SP1 Rollup Update 3.  

=======================================

Have a DAG?  Thinking about having a DAG?  I recommend you check out this awesome blog on designing a highly available database copy layout…

http://msexchangeteam.com/archive/2010/09/10/456211.aspx

This is a great blog post on Windows 2008 and Windows 2008 R2 Cluster Logs.

http://blogs.technet.com/b/askcore/archive/2010/04/13/understanding-the-cluster-debug-log-in-2008.aspx

TIMMCMIC

If an Exchange 2010 RTM server <or> an Exchange 2010 SP1 Beta has been upgraded to Exchange 2010 SP1 RTM administrators may experience an error when attempting to utilize the remove-mailboxdatabasecopy <or> add-mailboxdatabasecopy commandlets.

When running remove-mailboxdatabasecopy the following error is noted:

Remove-MailboxDatabaseCopy DAG-DB0DAG-2 –Verbose

WARNING: An unexpected error has occurred and a Watson dump is being generated: Registry key has subkeys and recursive removes are not supported by this method.

Registry key has subkeys and recursive removes are not supported by this method.
    + CategoryInfo          : NotSpecified: (:) [Remove-MailboxDatabaseCopy], InvalidOperationException
    + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.Exchange.Management.SystemConfigurationTasks. 
  RemoveMailboxDatabaseCopy

Although the error is reported, the remove was successful in updating the database object within the active directory to show the server no longer hosts a copy of the database.  You can verify the copy was successfully removed by reviewing the Servers with the get-mailboxdatabase –identity <NAME> | fl name, servers commandlet.

Here is sample output (note DAG-2 is missing):

[PS] D:>Get-MailboxDatabase DAG-DB0 | fl name,servers

Name    : DAG-DB0
Servers : {DAG-1, DAG-3, DAG-4}

If an administrator attempts to add a database copy to a DAG member, the same error may also be returned.

Add-MailboxDatabaseCopy DAG-DB0 -MailboxServer DAG-2

WARNING: An unexpected error has occurred and a Watson dump is being generated: Registry key has subkeys and recursive
removes are not supported by this method.

Registry key has subkeys and recursive removes are not supported by this method.
    + CategoryInfo          : NotSpecified: (:) [Add-MailboxDatabaseCopy], InvalidOperationException
    + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.Exchange.Management.SystemConfigurationTasks.
   AddMailboxDatabaseCopy

Unlike the remove-mailboxdatabasecopy this command is not successful in adding the copy <or> updating the Active Directory to show the copy was added.

To work around this issue the administrator should:

1)  Identify the GUID of the database that is being added.

2)  On the server specified in the add command, using the database GUID identified, remove the following registry key:

HKEY_LOCAL_MACHINESOFTWAREMicrosoftExchangeServerv14ReplayState{DB-GUID}DumpsterInfo

To identify the mailbox database GUID, use the following command:

[PS] D:>Get-MailboxDatabase DAG-DB0 | fl name,GUID

Name : DAG-DB0
Guid : 8d3a9778-851c-40a4-91af-65a2c487b4cc

The GUID identified in this case is 8d3a9778-851c-40a4-91af-65a2c487b4cc.  With this information we can no export and delete the DUMPSTERINFO key on the server where you are attempting to add the mailbox database copy.

Once the registry key is removed the add-mailboxdatabasecopy command will complete successfully and the database copy will be added.

A question that has come up a few times recently is why does the date modified timestamp on my Exchange databases not change (even though the database is mounted and functioning).  Specifically some administrators have been looking at this as an indicator of health on a passive database copy – which it is not.

The date modified timestamp will generally get updated on an Exchange database when one of two things happen:

1)  The EDB file size is extended in order to accommodate data that does not fit into whitespace that currently exists in the database.

2)  The database is dismounted and all open handles to the file are released.

Note that the modified time is not subject to change if the contents of the file are changed – for example if whitespace is utilized within the database for the storage of new messages etc the date modified will not change.

To show this I used my lab to generate some examples.  Here is a screen shot of a database that was mounted last on 8/3/2010.  The database screen shot was taken 8/8/2010 before 8:29 am edt.

Using the Exchange Management Console, I dismounted the database at 8:29 am edt on 8/8/2010.

You will note that the date modified changed to the time and date the dismount occurred.  I then used the Exchange Management Console to re-mount the database.

After remounting the database I noted that the time remained the same as in the previous screen shot.  I then took some test mailboxes with content, and moved them into the mailbox store.  You will note in this screen shot that both the size and date modified changed – in this case the database file was extended on the partition so the change was expected.

It is normal for an Exchange database to not show an updated date modified and this field should be used to judge the health or utilization of an Exchange database.

In Exchange 2007 there are two clustered installation models.  Some customers elect to utilize a clustered installation model based on shared storage – this is a single copy cluster installation.  In order to achieve site resiliency or provide for disaster recovery, some customers will implement a SAN based data replication solution. 

Recently I encountered a customer that was utilizing SAN based data replication and the single copy cluster installation model to provide their site resilient solution.  The installation encompassed a source cluster with single copy configuration and a target cluster with single copy cluster configuration.  Each clustered mailbox server was established utilizing a different name – for example Exchange-Main and Exchange-DR.  The physical disk resources that were assigned to each CMS instance represented the LUNs that were replicated between SANs.  When it was necessary to activate the solution databases would be marked as “Allow this database to be overwritten by a restore” and then mounted.  Mailboxes would be moved utilizing the move-mailbox –configurationOnly to restore client access to the replicated databases

This presented an interesting challenge for this customer when it came to deploying service packs.  When the same physical disk resources are utilized between clusters, only one set of the physical disk resources can be brought online.  This is because one SAN has a Read / Write setting and the other SAN has a Read Only setting.  Essentially an online attempt of the database instances of the CMS Exchange-DR would fail because their dependant physical disks could not be brought online (because they were read only).

When an /upgradeCMS is performed after upgrading the binaries on a clustered node, the resources are initially in an offline state.  As a completion of the upgradeCMS the setup process initiates an online to the cluster mailbox server group.  Should any resources fail to come online this is considered a failure of the upgrade.  The administrator performing the upgrade is notified that a failure occurred and the upgrade setup watermark persists in the registry.  Therefore it is necessary that the /upgradeCMS be allowed to complete.  In this case database instances could not be brought online because their associated storage could not be brought online due to the storage being Read Only.

In order to complete the upgrade process the following steps were utilized (utilizing my sample clustered mailbox server names).

• Following SAN vendor recommendations replication was suspended between the Exchange-Main and Exchange-DR. 
• Mark LUNs on the remote SAN as Read / Write (allowing Exchange-DR full access to storage).
• Databases on the secondary CMS were set to “Allow this database to be overwritten by a restore”
  • Get-MailboxDatabase –server Exchange-DR | Set-MailboxDatabase –allowfilerestore:$TRUE
• Complete the upgrade process on Exchange-Main which will fully bring resources online.
• Complete the upgrade process on Exchange-DR which will fully bring resources online.

At this point both Exchange-Main and Exchange-DR are online.  This means that the databases that were previously replicated to Exchange-DR are no longer equal to the databases that exist on Exchange-Main.  As a post upgrade step we need to do the following:

• Stop the resources on Exchange-DR.
• Mark replicated LUNS on the remote SAN as Read Only (preventing Exchange-DR access to the storage).
• Following SAN vendor recommendations re-establish replication between the source and remote SANs ensuring that the SOURCE SAN is utilized for data synchronization.

In this installation it was necessary to temporarily break and re-establish replication in order to complete the /upgradeCMS process.

Exchange 2007 SP3 adds the support for utilizing Windows 2008 R2 servers. 

In Exchange 2007 Cluster Continuous Replication (CCR) installations, all log shipping activity by default occurs over the “public” cluster interface.  When administrators desire to have log shipping activities occur over a “private” network or desire to implement multiple replication paths between nodes, continuous replication hostnames can be utilized.

More information on Exchange 2007 CCR clusters and continuous replication hostnames can be found at http://technet.microsoft.com/en-us/library/bb124521(EXCHG.80).aspx.

Prior to implementing a continuous replication host name the get-clusteredservermailboxstatus commandlet can be utilized to see the current names services replication.  Here is a sample output from a cluster not configured to utilize continuous replication hostnames.

Identity                        : MBX-3
ClusteredMailboxServerName      : MBX-3.domain.com
State                           : Online
OperationalMachines             : {NODE-1 <Active>, Node-2 <Quorum Owner>}
FailedResources                 : {}
OperationalReplicationHostNames : {node-1, node-2}
FailedReplicationHostNames      : {}
InUseReplicationHostNames       : {node-1, node-2}
IsValid                         : True
ObjectState                     : Unchanged

After establishing the pre-requisites necessary to utilize continuous replication hostnames, the hostnames creation is performed using the enable-continuousreplicationhostname shell command.  (http://technet.microsoft.com/en-us/library/bb690985(EXCHG.80).aspx)

When attempting to enable a replication hostname on a Windows 2008 R2 cluster, the following error may be displayed in the management shell.

[PS] C:>Enable-ContinuousReplicationHostName -TargetMachine Node-1 -HostName Node-1-Repl-A -IPv4Address 10.0.1.3

Confirm
Are you sure you want to perform this action?

Enabling continuous replication host name "Node-1-Repl-A".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help
(default is "Y"):a
Enable-ContinuousReplicationHostName : Enable-ContinuousReplicationHostNameNetw
ork configuration could not be completed.
At line:1 char:37
+ Enable-ContinuousReplicationHostName <<<<  -TargetMachine Node-1 -HostName Node-1-Repl-A -IPv4Address 10.0.1.3
    + CategoryInfo          : InvalidOperation: (:) [Enable-ContinuousReplicat
   ionHostName], NetworkConfigException
    + FullyQualifiedErrorId : C3F1320,Microsoft.Exchange.Management.SystemConf
   igurationTasks.EnableContinuousReplicationHostName

When reviewing Failover Cluster Manager, the replication host name group containing the correct network name and ipv4 address appear to have been created successfully.

Although the continuous replication hostname group was created, reviewing get-clusteredservermailboxstatus indicates the name is not being utilized by the replication service on the cluster.

Identity                        : MBX-3
ClusteredMailboxServerName      : MBX-3.domain.com
State                           : Online
OperationalMachines             : {NODE-1 <Active>, Node-2 <Quorum Owner>}
FailedResources                 : {}
OperationalReplicationHostNames : {node-1, node-2}
FailedReplicationHostNames      : {}
InUseReplicationHostNames       : {node-1, node-2}
IsValid                         : True
ObjectState                     : Unchanged

When the replication service first starts up <or> the configuration time expires the replication service enumerates all network names on the cluster to determine which are valid endpoints for log shipping.  This is initially based on two cluster private properties stamped on each name, MSExchange_NetName and MSExchange_UseNetworkForLogCopying.  Each of these should have a value of 1 on a network name utilized as a continuous replication host name.

Listing private properties for ‘Network Name (Node-1-Repl-A)’:

T  Resource             Name                           Value

— ——————– —————————— ———————–

BR Network Name (Node-1-Repl-A) ResourceData                   01 00 00 00 … (260 bytes)

DR Network Name (Node-1-Repl-A) StatusNetBIOS                  0 (0x0)

DR Network Name (Node-1-Repl-A) StatusDNS                      0 (0x0)

DR Network Name (Node-1-Repl-A) StatusKerberos                 0 (0x0)

SR Network Name (Node-1-Repl-A) CreatingDC                     \DC-1.domain.com

FTR Network Name (Node-1-Repl-A) LastDNSUpdateTime              7/11/2010 2:26:26 PM

SR Network Name (Node-1-Repl-A) ObjectGUID                     5adc38b3281a004788f2a3e27ae7a0ce

S  Network Name (Node-1-Repl-A) Name                           NODE-1-REPL-A

S  Network Name (Node-1-Repl-A) DnsName                        Node-1-Repl-A

D  Network Name (Node-1-Repl-A) RemapPipeNames                 0 (0x0)

D  Network Name (Node-1-Repl-A) HostRecordTTL                  1200 (0x4b0)

D  Network Name (Node-1-Repl-A) RegisterAllProvidersIP         0 (0x0)

D  Network Name (Node-1-Repl-A) PublishPTRRecords              0 (0x0)

D  Network Name (Node-1-Repl-A) TimerCallbackAdditionalThreshold 5 (0x5)

D  Network Name (Node-1-Repl-A) MSExchange_NetName             1 (0x1)

D  Network Name (Node-1-Repl-A) RequireDNS                     1 (0x1)

D  Network Name (Node-1-Repl-A) MSExchange_UseNetworkForLogCopying 1 (0x1)

On the surface it would appear that there is nothing preventing this name from operating correctly as a continuous replication host name.  After performing some internal tracing it was determined that the replication service is also implementing another check on a network name resource to ensure that it can be satisfactorily utilized for replication – is Kerberos enabled for the network name.  The replication service performs this check by reviewing a private property of a network name resource – requirekerberos and ensuring it has a value of 1.

In Windows 2003 network name resources could be enabled for Kerberos at the administrators discretion.  In Windows 2008 and Windows 2008 R2 all network names must be Kerberos enabled.  In Windows 2008 requireKerberos is a valid private property and can be programatically set.  In Windows 2008 R2 the requireKerberos property has been deprecated and can be no longer be programmatically set.  Without the requireKerberos property in Windows 2008 R2 the enable-continuousreplicationhostname commandlet fails with the previously documented error. 

To work around this issue and allow the replication host names created with the enable-continuousreplicationhostname command to function the following steps can be performed:

• Using the Exchange Managment Shell invoke the enable-continuousreplicationhostname command.  Allow the command to create the resource group, network name, and IPv4 resource. 
• Verify with Failover Cluster Manager that the resource group, network name, and IPv4 resource were created and are online.
• Manually set requireKerberos utilizing either cluster.exe or Failover Cluster Powershell extensions (preferred)
  • Cluster.exe
    • Set the requirekerberos key.
      • Cluster.exe <clusterFQDN> res "<Network Name> /priv requirekerberos=1:DWORD
      • Example:  cluster.exe cluster cluster-1.domain.com res “Network Name (Node-1-Repl-A)” /priv requirekerberos=1:DWORD
      • Note that requirekerberos is all lowercase.
    • Take offline and online the continuous replication hostname group.
      • Cluster.exe <clusterFQDN> group <Group> /offline
      • Example:  cluster.exe cluster.domain.com group “Node-1-Repl-A_group” /offline
      • Cluster.exe <clusterFQDN group <Group> /online
      • Example:  cluster.exe cluster.domain.com group “Node-1-Repl-A_group” /online
    • Restart the replication service
      • net stop msexchangerepl
      • net start msexchangerepl
  • PowerShell
    • Import the failover cluster powershell extensions.
      • Import-Module FailoverClusters
    • Set the requirekerberos key.
      • Get-ClusterResource <Network Name> | Set-ClusterParameter requirekerberos 1
      • Example:  Get-ClusterResource “Network Name (Node-1-Repl-A)” | Set-ClusterParameter –create requirekerberos 1
      • Node that requirekerberos is all lowercase.
    • Take offline and online the continuous replication hostname group.
      • Stop-ClusterGroup –cluster <ClusterFQDN> –Name <Group>
      • Example:  Stop-ClusterGroup –cluster Cluster.domain.com –Name Node-1-Repl-A_group
      • Start-ClusterGroup –cluster <ClusterFQDN> –Name <Group>
      • Example:  Start-ClusterGroup –cluster Cluster.domain.com –Name Node-1-Repl-A_group
    • Restart the replication service.
      • Stop-Service msexchangerepl
      • Start-Service msexchangerepl

At this time you can utilize either cluster.exe or powershell to verify that the requirekerboros key has been created with a value of 1.

Cluster.exe <clusterFQDN> res <Network Name> /priv  –> Cluster.exe cluster.domain.com res “Network Name (Node-1-Repl-A)” /priv

Listing private properties for ‘Network Name (Node-1-Repl-A)’:

T  Resource             Name                           Value

— ——————– —————————— ———————–

BR Network Name (Node-1-Repl-A) ResourceData                   01 00 00 00 … (260 bytes)

DR Network Name (Node-1-Repl-A) StatusNetBIOS                  0 (0x0)

DR Network Name (Node-1-Repl-A) StatusDNS                      0 (0x0)

DR Network Name (Node-1-Repl-A) StatusKerberos                 0 (0x0)

SR Network Name (Node-1-Repl-A) CreatingDC                     \DC-1.domain.com

FTR Network Name (Node-1-Repl-A) LastDNSUpdateTime              7/11/2010 2:26:26 PM

SR Network Name (Node-1-Repl-A) ObjectGUID                     5adc38b3281a004788f2a3e27ae7a0ce

S  Network Name (Node-1-Repl-A) Name                           NODE-1-REPL-A

S  Network Name (Node-1-Repl-A) DnsName                        Node-1-Repl-A

D  Network Name (Node-1-Repl-A) RemapPipeNames                 0 (0x0)

D  Network Name (Node-1-Repl-A) HostRecordTTL                  1200 (0x4b0)

D  Network Name (Node-1-Repl-A) RegisterAllProvidersIP         0 (0x0)

D  Network Name (Node-1-Repl-A) PublishPTRRecords              0 (0x0)

D  Network Name (Node-1-Repl-A) TimerCallbackAdditionalThreshold 5 (0x5)

D  Network Name (Node-1-Repl-A) MSExchange_NetName             1 (0x1)

D  Network Name (Node-1-Repl-A) RequireDNS                     1 (0x1)

D  Network Name (Node-1-Repl-A) MSExchange_UseNetworkForLogCopying 1 (0x1)

D  Network Name (Node-1-Repl-A) requirekerberos                1 (0x1)

Get-ClusterResource <NAME> | Get-ClusterParameter

Object              Name                Value               Type              
——              —-                —–               —-              
Network Name (No… Name                NODE-1-REPL-A       String            
Network Name (No… DnsName             Node-1-Repl-A       String            
Network Name (No… RemapPipeNames      0                   UInt32            
Network Name (No… HostRecordTTL       1200                UInt32            
Network Name (No… RegisterAllProvi… 0                   UInt32            
Network Name (No… PublishPTRRecords   0                   UInt32            
Network Name (No… TimerCallbackAdd… 5                   UInt32            
Network Name (No… MSExchange_NetName  1                   UInt32            
Network Name (No… RequireDNS          1                   UInt32            
Network Name (No… MSExchange_UseNe… 1                   UInt32            
Network Name (No… requirekerberos     1                   UInt32            
Network Name (No… ResourceData        {1, 0, 0, 0, 118… ByteArray         
Network Name (No… StatusNetBIOS       0                   UInt32            
Network Name (No… StatusDNS           0                   UInt32            
Network Name (No… StatusKerberos      0                   UInt32            
Network Name (No… CreatingDC          \DC-1.domain…… String            
Network Name (No… LastDNSUpdateTime   7/11/2010 9:26:2… DateTime          
Network Name (No… ObjectGUID          5adc38b3281a0047… String            

By restarting the replication service after setting this key the replication services configuration is immediately updated.  At this time the replication service should detect and begin to utilize the replication hostnames created.  This can be verified using the get-clusteredservermailboxstatus commandlet.

Identity                        : MBX-3
ClusteredMailboxServerName      : MBX-3.exchange.msft
State                           : Online
OperationalMachines             : {NODE-1 <Active>, Node-2 <Quorum Owner>}
FailedResources                 : {}
OperationalReplicationHostNames : {node-1-repl-a, node-1, node-2}
FailedReplicationHostNames      : {}
InUseReplicationHostNames       : {node-1-repl-a, node-2}
IsValid                         : True
ObjectState                     : Unchanged

At this time we are investigating a fix that does not require a workaround.  As changes occur I will update this blog.

In Exchange 2010 when a Database Availability Group (DAG) it utilized, and there is an even number of DAG members, the underlying cluster is implemented utilizing the quorum type Node and File Share Majority.  The settings utilized for the File Share Witness are defined on the DAG when the logical DAG object is created and are either set by the administrator or automatically defined.

To verify the quorum type you can use either cluster.exe or cluster powershell extensions (Preferred)

Cluster.exe <cluster> /quorum  (Windows 2008 & Windows 2008 R2)

Cluster.exe cluster.domain.com /quorum

Witness Resource Name Path                                          Type

——————— ——————————————— ——–

File Share Witness (\HT-1.DOMAIN.COMDAG.DOMAIN.COM)               Majority

Get-Cluster <cluster> | Get-ClusterQuorum | FL (Windows 2008 R2 Only)

Cluster        : DAG
QuorumResource : File Share Witness (\HT-1.DOMAIN.COMDAG.DOMAIN.COM)
QuorumType     : NodeAndFileShareMajority

In Failover Cluster Manager, the resources can be viewed by looking at the Cluster Core Resources.

It may become necessary to change the server hosting the file share witness.  In Exchange 2010 this is not done utilizing Failover Cluster Manager, but rather utilizing the set-databaseavailabilitygroup commandlet.  It is after the witness server is successfully updated that the oddity occurs.  Here’s an example:

Currently the DAG utilizes the witness server HT-1.  Using the set-databaseavailabilitygroup command the witness server is changed to HT-2.  (set-databaseavailabilitygroupserver –witnessServer HT-2)  The command returns without error.  When running the previous cluster commands the following output is noted:

Cluster.exe cluster.domain.com /quorum (Windows 2008 and Windows 2008 R2)

Witness Resource Name Path                                          Type

——————— ——————————————— ——–

File Share Witness (\HT-1.DOMAIN.COMDAG.DOMAIN.COM)               Majority

Get-Cluster <cluster> | Get-ClusterQuorum | FL (Windows 2008 R2 Only)

Cluster        : DAG
QuorumResource : File Share Witness (\HT-1.DOMAIN.COMDAG.DOMAIN.COM)
QuorumType     : NodeAndFileShareMajority

Also in Failover Cluster Manager the following is noted in the cluster core resources group.

After looking at this output the administrator could be lead to believe that the witness server did not successfully update.  After all both cluster.exe and powershell both show the File Share Witness (\HT-1.DOMAIN.COMDAG.DOMAIN.COM).  It is only in Failover Cluster Manager, if the windows is fully expanded, that you can see both (\HT-1.DOMAIN.COMDAG.DOMAIN.COM) and (\HT-2.DOMAIN.COMDAG.DOMAIN.COM).  This leads administrators to believe that two file share witness servers are currently in use.

Thankfully both of these perceived conditions are false.  The command was both successful in changing the witness server and only one file share witness is in use.

Each cluster resource has a display name and a set of public and private properties.  Unfortunately when using set-databaseavailabilitygroup to change the witness server, the File Share Witness resource private property for where the witness is stored is updated but the public property display name, which contains the previous witness server, is not.  Let’s take a look at this further.

Using cluster.exe or powershell I can review the private properties of the File Share Witness resource.  (Command output truncated to show relevant values only.)

Cluster.exe <cluster> res <resource> /priv <or> /prop (Windows 2008 & Windows 2008 R2)

Cluster.exe cluster.domain.com res “File Share Witness (\HT-1.domain.comDAG.domain.com)" /prop

Listing properties for ‘File Share Witness (\HT-1.domain.COMDAG.domain.COM)’:

T  Resource             Name                           Value

— ——————– —————————— ———————–

SR File Share Witness (\HT-1.domain.COMDAG.domain.COM) Name                           File Share Witness (\HT-1.domain.COMDAG.domain.COM)

Cluster.exe cluster.domain.com res “File Share Witness (\HT-1.domain.comDAG.domain.com)" /priv

Listing private properties for ‘File Share Witness (\HT-1.domain.COMDAG.domain.COM)’:

T  Resource             Name                           Value

— ——————– —————————— ———————–

S  File Share Witness (\HT-1.domain.COMDAG.domain.COM) SharePath                      \HT-1.domain.comDAG.domain.com

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | fl (Windows 2008 R2 Only – Public Properties)

Name         : File Share Witness (\HT-1.domain.COMDAG.domain.COM)
State        : Online
OwnerGroup   : Cluster Group
ResourceType : File Share Witness

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | Get-ClusterParameter fl (Windows 2008 R2 Only – Private Properties)

Name          : SharePath
IsReadOnly    : False
ParameterType : String
Value         : \HT-1.domain.comDAG.domain.com

At this time a set-databaseavailability group is issued to change the witness server.  After the command completes successfully, the previous commands are run.  (Command output truncated to show relevant values only.)

Cluster.exe cluster.domain.com res “File Share Witness (\HT-1.domain.comDAG.domain.com)" /prop

Listing properties for ‘File Share Witness (\HT-1.domain.COMDAG.domain.COM)’:

T  Resource             Name                           Value

— ——————– —————————— ———————–

SR File Share Witness (\HT-1.domain.COMDAG.domain.COM) Name                           File Share Witness (\HT-1.domain.COMDAG.domain.COM)

Cluster.exe cluster.domain.com res “File Share Witness (\HT-1.domain.comDAG.domain.com)" /priv

Listing private properties for ‘File Share Witness (\HT-1.domain.COMDAG.domain.COM)’:

T  Resource             Name                           Value

— ——————– —————————— ———————–

S  File Share Witness (\HT-1.domain.COMDAG.domain.COM) SharePath                      \HT-2.domain.comDAG.domain.com

(Note:  The SharePath in the previous output reflects the new witness server as expected)

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | fl (Windows 2008 R2 Only – Public Properties)

Name         : File Share Witness (\HT-1.domain.COMDAG.domain.COM)
State        : Online
OwnerGroup   : Cluster Group
ResourceType : File Share Witness

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | Get-ClusterParameter fl (Windows 2008 R2 Only – Private Properties)

Name          : SharePath
IsReadOnly    : False
ParameterType : String
Value         : \HT-2.domain.comDAG.domain.com

(Note:  The SharePath in the previous output reflects the new witness server as expected)

As you can see the set-databaseavailability group command did complete it’s task successfully by updating the SharePath attribute of the quorum resource to utilize the correct witness server.

The use of mount points for Exchange is becoming more common place in many installations.  Some customers feel the best implementation of mount points consists of a small root disk with mount points created from folders on that disk.

For example, I may have a Drive L: that is 10 megs and I may create 4 folders on this drive (Database1 / Database2 / Database3 / Database4).  I will then create mount points utilizing the folders created from the L drive.

There are certain process in Exchange that often check for free drive space prior to performing certain operations.  Unfortunately these processes are not necessarily mount point aware – therefore they end up querying the free drive space of the lettered volume rather than the mount point.  One of these process is MSSearch.

MSSearch by default creates a catalog data folder co-located with each EDB file.  In our example above the catalog data folder and the edb file would be in L:Database1 (where Database1 is the mount point).  In this this case the L drive has 10 megs free space but the Database1 mount point has 1.5 terabytes of free space.  When MSSearch attempts to initialize the initial catalog this operation fails as the drive space reported by the disk L is not sufficient (even though there is plenty of space where the actual catalog is stored).

Here is an example of some events you may see when this occurs.

Log Name:      Application
Source:        MSExchange Search Indexer
Date:          6/14/2010 12:11:20 PM
Event ID:      104
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server.company.com
Description:
Exchange Search Indexer failed to enable the Mailbox Database DATABASE(GUID = 58c0ed8a-dbfc-4d55-b265-8a80f1dc477b) after 1 tries. The last failure was: System.ComponentModel.Win32Exception: Unable to SetProperty FTE_PluginList on catalog ExSearch-58c0ed8a-dbfc-4d55-b265-8a80f1dc477b-26fc1c62-d3e8-4711-b3c9-3bb0b32aec0a. Error = -2147215320
   at Microsoft.Exchange.Msfte.CFTEAdmin.SetProperty(CatalogState catalogInfo, PropertyScope propertyScope, String propertyName, Object propertyValue, Boolean throwOnFailure)
   at Microsoft.Exchange.Msfte.CFTEAdmin.CreateCatalog(CatalogState catalogInfo)
   at Microsoft.Exchange.Search.Globals.CreateCatalog(CatalogState state, String reason)
   at Microsoft.Exchange.Search.Globals.RecreateCatalogAndPropertyStore(CatalogState catalogInfo, String reason)
   at Microsoft.Exchange.Search.CatalogState.CreateNew(String reason)
   at Microsoft.Exchange.Search.CatalogState.Reset(String reason)
   at Microsoft.Exchange.Search.CatalogState.HandleMountCatalogException(Exception exception)
   at Microsoft.Exchange.Search.Globals.CheckAndInitializeCatalog(CatalogState catalogInfo)
   at Microsoft.Exchange.Search.Driver.ProcessNewCatalogInternal(CatalogState catalog, List`1 mdbsToCrawl, Int32& numberOfDisabledMDBs). It will retry after 10 minutes.

Log Name:      Application
Source:        ExchangeStoreDB
Date:          6/14/2010 12:12:51 PM
Event ID:      222
Task Category: Database recovery
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server.company.com
Description:
At ‘6/14/2010 11:12:50 AM’ the Microsoft Exchange Information Store Database ‘DATABASE’ copy on this server experienced a corrupted search catalog. The error returned by failover was "There is only one copy of this mailbox database (DATABASE). Automatic recovery is not available.". Consult the event log on the server for other "ExchangeStoreDb" and "MSExchange Search Indexer" events for more specific information about the failures.

The important information is actually contained in the first event – the error code –2147215320.  This error code translates to CI_E_CONFIG_DISK_FULL.

To resolve this issue you can:

• Increase the space allotted to the root disk hosting the mount point.
• Change from utilizing mount points to drive letters.

Once this is done restarting the MSSearch services may be necessary so that initial catalog creation can occur.