Recently I faced an issue with Exchange 2013 server in production environment. After a failed update of Exchange 2013 to CU5 the mail flow stopped working and sending emails stuck in drafts folder. Even though I managed to recover Exchange application and user mail boxes, but mail flow of internal and external did not work.Even sending mails from smart phones went to draft folder in the mail box server where I can see that in Outlook web access. After two days of continues research, trial and error I managed to make it work without a new installation of Exchange server.
Domain controller and Exchange servers are running on Windows 2012 R2 as virtual machine of Hyper-V. Installed version was first release of Exchange and mail flow was working fine till the update crash. The SP1 (CU4) or CU5 updates are recommended for for customers who are using first release which came 2 years ago. Also there was some issues on authentication and outlook anywhere which looked like bug on the older version of Exchange 2013 and recommended fix was to update latest CU5 release.
Backup is always important. Even though we have enterprise backup solution, somehow its not working with Windows 2012 R2 virtual machines and Exchange 2013 application. So, I have used Windows 2012 R2 Windows server backup to take full backup of Exchange 2013 server (including system state and Information store) and Domain controller. Also, I took the system state back of these two servers separately.
What Went Wrong
The Exchange server had enough space inside C drive before start the upgrade process. But I missed to check the space on storage LUN where the virtual machine VHD file located.
The upgrade process started, prepared organization, removed existing Exchange 2013 installation and started copying new files. BOOM, the virtual machine went to pause mode with ugly Windows blue screen error ( now changed in Windows 2012 and Windows 8).
I made some space on the LUN and resumed the server. Unfortunately it got restarted with a blank Windows without new or old Exchange installation, because old installation was removed during the upgrade process.
How I fixed Exchange 2013 Emails Stuck in Drafts After Failed CU or SP Update
I’m not going to share all steps and attempts I tries to fix the issues, but here are the important notes.
1) Windows server backup/restore for Exchange 2013 works fine. If you have taken full Exchange 2013 backup as shown here in Microsoft site, the restore will work fine.
2) Always keep the full and latest backup of Domain controller ( Active directory) an Exchange server before update.
3) There are few DNS settings need to be done to send and receive emails internally and externally at first time when Exchange 2013 server configured. I’m sure you would have done them initially that’s why the mail flow worked fine till the update issue.
So, if you do an Exchange application restore after any issues created by failed CU or SP update, you do not need to change anything on Exchange server settings in ECP control panel. Because the application restore would restore exact settings again.
You can refer our earlier guide to make sure these DNS and hosts files settings are there.
4) There is noting you need to do in server firewall or main hardware firewall settings (to allow or deny IPs and port forwards), because we have not touched them too.
5) Mostly you do not need to change any settings in Exchange server if your backup and restore are perfect.
6) So, the culprit is domain controller Active Directory data, yes, that’s right.
When the Exchange 2013 CU5 update happens, it modifies the AD schema in domain controller to match with the version its going to install. In my case it already changed to the version of CU 5.
When I restored the Exchange server to earlier backup, it was for the older schema version. Therefore some kind of authentication or compatibility issue were there with the new Exchange schema in AD and old Exchange server version.
Here is the way to find out the Exchange server schema version in domain controller.
Here is server version and schema numbers. You can visit the original link for more updates. You should not change these numbers in your active directory by ADSIEdit tool. It’s highly recommended to NOT to touch and modify AD data manually.
The solution is either restore your AD data from backup (you have to think how it will affect your other domain controllers and clients) or update the Exchange 2013 again which is matching the schema number.
In my case, I made enough space in virtual machine’s LUN and ran the CU 5 update again. Without touching any settings in Exchange server or ECP, mail flow started working, active sync and Outlook anywhere all came back online.
You can verify the server health and status after the upgrade by reading this great guide about how to update Exchange 2013 to SP1.
I’m sure this post would be helpful to someone who struggles with Exchange 2013 sending mails go to draft folder after failed service pack or CU (Cumulative Updates).