Exchange 2010 – Reseeding failed Database with multithreading

Reseeding is a process of fixing the failed passive copy of the database which basically mean is, the passive database copy is out of sync with active database. Passive copies can be a failed database or failed Index. When Database goes in failed state or failed and suspended state or database Index goes in to failed state then it needs administrator intervension and force the database reseed.  

There can be various reason for database to fail. Replication service running on the host machine is responsible for keeping the database in healthy state. It tries to take corrective action if the database goes out of sync else administrator may have to fix failed database manually.

Below is the nice piece of code which will request you to enter the DAG Name and it will determine the list of failed database and perform full reseed on each of the failed database. In the normal process, reseeding happends on the single database at a time and you can’t limit how many database you can reseed at a time. Eg

Get-MailboxDatabaseCopyStatus $strResponse  |?{$_.status -like “Failed*”} | update-mailboxdatabasecopystatus -deleteexistingfiles -confirm:$false

In the above example cmdlet will get all the failed database and it pipes to update-mailboxdatabase cmdlet. Update-Mailboxdatabasecopy performs the full reseed of the failed database one by one and brings the database into healthy state. If we have very bigger database like 100 GB and it has to update to different site then you know how long it may take. With this senarion you dont want to fix one failed database at a time.

Below script helps you to address the above defined issue. It can reseed the failed database up to max of 10 database in 10 different window at a time and if one database reseeding completes then new failed database will reseed if there is any. This count can be reduced or increased based on the performance of the local server and the network available.

function Createfolders()
{
 remove-item -path "C:\DBs\bt" -force  -Recurse -confirm:$false -ErrorAction SilentlyContinue | out-null
 remove-item -path "C:\DBs\ps" -force  -Recurse -confirm:$false -ErrorAction SilentlyContinue| out-null
 remove-item -path "C:\DBs" -force  -Recurse -confirm:$false -ErrorAction SilentlyContinue| out-null

 new-item -path "C:\DBs" -ItemType Directory -force | out-null
 new-item -path "C:\DBs\bt" -ItemType Directory -force | out-null
 new-item -path "C:\DBs\ps" -ItemType Directory -force | out-null
}
$strResponse = Read-Host  "`nPlease enter DAG Name to reseed the failed Databases"

write-host -f Magenta "Checking for Failed Database copies in the DAG : $strResponse"

$databases = Get-MailboxDatabaseCopyStatus $strResponse  |?{$_.status -like "Failed*"}
if($databases -ne $null)
{
 write-host -f red "Following Databases are in failed state"
 $databases
 Write-host "`n"
 foreach($database in $databases)
 {
 $filename = $database.name
 $dbname = $database.name

 $filename = $filename.Replace("\", "_")
 $DBcopyReport1 = "C:\DBs\bt\$filename.bat"
 $DBcopyReport2 = "C:\DBs\ps\$filename.ps1"
 New-item -ItemType file -path $DBcopyReport1 -force | out-null
 New-item -ItemType file -path $DBcopyReport2 -force | out-null

 "Powershell.exe `"C:\DBs\bt\$filename.ps1`"" |  Out-File -filepath $DBcopyReport1 -encoding ASCII -append
 "Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010 -ErrorAction SilentlyContinue"| Out-file $DBcopyReport2 -encoding ASCII -append
 "Suspend-MailboxDatabaseCopy -Identity `"$dbname`" -confirm:" + "$" + "false"| Out-file $DBcopyReport2 -encoding ASCII -append
 "Update-MailboxDatabaseCopy -Identity `"$dbname`"  -DeleteExistingFiles -confirm:" + "$" + "false -ErrorAction:Stop -WarningAction:SilentlyContinue" | Out-file $DBcopyReport2 -encoding ASCII -append

 

 $files = [IO.Directory]::GetFiles("C:\DBs\bt\")
 $cmdprocess = @()
 Write-host -f yellow "`nReseeding the following databases"
 for ($i=0; $i -lt $files.count; $i++)
 {
  $DBDName = $files[$i]
  $DBDName =$DBDName.split("\")[3]
  $DBDName =$DBDName.split(".")[0]
  $DBDName = $DBDName.Replace("_","\")
  Write-host -f yellow "$DBDName"
  $cmdprocess =$cmdprocess+ [diagnostics.process]::Start($files[$i])
  do
  { 
   $cmdp = @()
   $continue = 0
   foreach($cmdproces in $cmdprocess)
   {
    $cmdp = $cmdp + $cmdproces.id
   }
   $processid = Get-Process | %{$_.id}
   foreach($cmd in $cmdp)
   {
    if($processid -contains $cmd)
    {  
     $continue = $continue + 1
    }
   }
   start-sleep(10)
 
  }until($continue -lt 10)

 }
 do
 {
  $processid = Get-Process | %{$_.id}
  $Loopexit = 0
  foreach($cmd in $cmdp)
  { 
   if($processid -contains $cmd)
   {
   $Loopexit = 1
   start-sleep(10)
   }
  }
 }
 until($Loopexit -eq 0)

 Write-host -f Green "`nReseeding of Failed DB's has been completed"
}
Else
{
 Write-host -f Green "All the mailbox Database copy are in Healthy state"
}

Createfolders
Write-host -f Magenta "`nChecking for failed Catalog or Content Index in the DAG :$strResponse"
$databases = Get-MailboxDatabaseCopyStatus $strResponse  |?{$_.ContentIndexState -match "Fail" }
if($databases -ne $null)
{
write-host -f red "Following Databases are in failed state"
 $databases
 Write-host "`n"
 foreach($database in $databases)
 {
 $filename = $database.name
 $dbname = $database.name

 $filename = $filename.Replace("\", "_")
 $DBcopyReport1 = "C:\DBs\bt\$filename.bat"
 $DBcopyReport2 = "C:\DBs\ps\$filename.ps1"
 New-item -ItemType file -path $DBcopyReport1 -force | out-null
 New-item -ItemType file -path $DBcopyReport2 -force | out-null

 "Powershell.exe `"C:\DBs\bt\$filename.ps1`"" |  Out-File -filepath $DBcopyReport1 -encoding ASCII -append
 "Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010 -ErrorAction SilentlyContinue"| Out-file $DBcopyReport2 -encoding ASCII -append
 "Suspend-MailboxDatabaseCopy -Identity `"$dbname`" -confirm:" + "$" + "false"| Out-file $DBcopyReport2 -encoding ASCII -append
 "Update-MailboxDatabaseCopy -Identity `"$dbname`"  -DeleteExistingFiles -confirm:" + "$" + "false -ErrorAction:Stop -WarningAction:SilentlyContinue" | Out-file $DBcopyReport2 -encoding ASCII -append

 

 $files = [IO.Directory]::GetFiles("C:\DBs\bt\")
 $cmdprocess = @()
 Write-host -f yellow "`nReseeding the following databases"
 for ($i=0; $i -lt $files.count; $i++)
 {
  $DBDName = $files[$i]
  $DBDName =$DBDName.split("\")[3]
  $DBDName =$DBDName.split(".")[0]
  $DBDName = $DBDName.Replace("_","\")
  Write-host -f yellow "$DBDName"
  $cmdprocess =$cmdprocess+ [diagnostics.process]::Start($files[$i])
  do
  { 
   $cmdp = @()
   $continue = 0
   foreach($cmdproces in $cmdprocess)
   {
    $cmdp = $cmdp + $cmdproces.id
   }
   $processid = Get-Process | %{$_.id}
   foreach($cmd in $cmdp)
   {
    if($processid -contains $cmd)
    {  
     $continue = $continue + 1
    }
   }
   start-sleep(10)
 
  }until($continue -lt 10)

 }
 do
 {
  $processid = Get-Process | %{$_.id}
  $Loopexit = 0
  foreach($cmd in $cmdp)
  { 
   if($processid -contains $cmd)
   {
   $Loopexit = 1
   start-sleep(10)
   }
  }
 }
 until($Loopexit -eq 0)

 Write-host -f Green "`nReseeding of Failed DB's has been completed"
}
Else
{
 Write-host -f Green "All the mailbox Database copy Index are in Healthy state"
}

Below is the snap of the execution window and we can see how DB reseed is been executed on multiple window. This will save you lot of time and effors in fixing the database. I hope this article will be helpful to you 🙂

Powershell Script to generate report on update rollup installed on all the Exchange Server 2010 Servers

Latest RU available at this point of time is RU5. When you start updating it’s hard to keep track which server is updated and which server is not unless you make note off. If you wanted to query all the exchange 2010 to find what the rollup version is installed then it’s not easy to find one. Exchange management shell and console will not show the version number of the RU, it only show the major version and service pack version. Exchange console and shell gets updated when ever we install server pack. So one of the best ways to find the latest RUs installed is by checking the version number of Exsetup.exe which is available in the Bin director on the Exchange installed servers.

Below is the script which queries all the Exchange 2010 serves for the file version of Exsetup.exe and write to the text file result.txt. It has the data in the format servername, version number and role installed and each is divided by “|” so its easy to format using excel

$installpath = "c$\Program Files\Microsoft\Exchange Server\V14\Bin\ExSetup.exe"
Get-ExchangeServer  | ?{$_.admindisplayversion -like  "*14*"} | %{
$Servername = $_.Name
$role = $_.serverrole
$Path = "\\" + $Servername + "\" + $installpath
$fileversion = (Get-Command $Path).FileVersionInfo |ForEach {$_.FileVersion}
$result = $Servername + "|" +  $fileversion + "|" + $role
$result
$result >> result.txt
}

I hope this helps you some day to pull this report for your team and manager real quick 🙂

Enable and Disabling Circular logging on Exchange 2010 DAG Database on a fly.

In Exchange 2007 and earlier version of Exchange if you wanted to enable or disable circular logging then you should dismount and mount the database to bring this into effect. In Exchange 2010 if you enable or disable circular logging then this chance will come into effect with in 30 seconds. This is done with the help of replication service and you dont have to dismount and mount the active database to get this changes applied. I think this is cool. You can make the changes on the fly without disrupting the mailbox live users.

Enable circular logging on a Particular database

Get-mailboxdatabase -Identity DBname | set-mailboxdatabase -CircularloggingEnabled $true

Disable circular logging on a Particular database

Get-mailboxdatabase -Identity DBname | set-mailboxdatabase -CircularloggingEnabled $false

Enable Circular logging on all the Database in a perticular DAG

Get-mailboxdatabase -identity DAGname* | set-mailboxdatabase -CircularloggingEnabled $true

Disable Circular logging on all the database in a perticular DAG

Get-mailboxdatabase -identity DAGname* | set-mailboxdatabase -CircularloggingEnabled $false

Feeling nice to blog after a long time on a auspecious day 🙂