Troubleshooting Offline Address Book Generation on Exchange 2010

http://blogs.catapultsystems.com/IT/archive/2010/03/11/troubleshooting-offline-address-book-generation-on-exchange-2010.aspx

Không còn PST nữa...haha....còn lâu...huhu

Mất hơn 2 tuần migrate Exchange 2003 sang 2010 xong và háo hức setup các hot features của Exchange 2010 để giới thiệu với bà con gần xa. Một trong số đó là Online Archive. Không còn pst file nữa, quá đã...haha.
Cho dù phải bấm bụng order thêm Enterprise CAL thì feature này cũng đáng đồng tiền bát gạo. Tạo riêng một mailbox db, enable online archive, vô OWA test thử, NOKIA..:-)...uống 1 ly cafe cho tỉnh táo cái đã để chuẩn bị document và email cho các users thì..... phát hiện nó chưa hề xuất hiện trong Outlook 2010 của mình, bộ Office 2010 Std mới implemented vài tháng trước....hmm
Đào bới các website về licensing thì té ngửa ra rằng feature này chỉ xuất hiện khi sử dụng Office 2010 phiên bản Pro Plus mà chỉ có em Volume Licensing mới có thoi nhehoặc phải sử dụng Outlook OEM or retail...:-(.
Má ơi đúng là lừa đảo kiểu Bill Gates cho dù ....Bill đã nghỉ hưu lâu rồi.
Nếu bạn là dân hay sử dụng đồ chùa thì không nói làm gì nhưng đã là dân sành điệu đã bỏ tiền ra mua chịu hàng hiệu :-) mà còn bị lừa thì đúng là đau còn hơn bò đá :-(. Đúng là không thể trách được tại sao người ta hay dùng crack...
Chính sách licence của Bill đã đủ phức tạp lắm lắm rồi mà còn cố ý cheat các khách hàng của mình nữa...bó tay thật...

Exchange 2010 mailboxes can not send to 2003 mailboxes

PROBLEM
Now we installed a new Exchange 2010 server in the same domain as the exchange 2003 server and installed the CAS, Mailbox and Hub roles. Next we about to move 70 mailboxes to the new Exchange 2010 server. Everything looks fine, users with mailboxed on the Exchange 2003 server can send mail to users on Exchange 2010 server. But the users with mailboxes on Exchange 2010 server can't send mail to mailboxes on exchange 2003. These messages stay in the 'SmtpRelayToTiRg' queue with error : 451 4.4.0 Primary Target IP address responded with: "451 5.7.3 Cannot Achieve Exchange Server authentication" 
~~~~~

SOLUTION 1
I installed a Windows 2007 Exchange server in to my 2003 environment this week. All went well apart from that the mail sending from the 2007 test mailboxes ended up in a queue called smtprelaytotirg. The error message given being 451 4.4.0 Primary Target IP Address Responded with (501 5.5.4 Auth Command Cancelled).
This queue is basically where 2007 is failing to deliver because it can’t route correctly.
The resolution in this case for me was easy. All the connectors were in place as they should be, but the transport for SMTP on the original master 2003 Exchange server had limited access to certain IP addresses.
I added the IP address for the new Exchange 2007 server and bang, 20 mins later the queue is empty.
Just a minor hurdle :) Oh, the other one to watch out for is making sure it can resolve either by IP or FQDN for the SMTP server, it’ll fail on netbios or just a single name. Anyone still relying on WINS needs shot in the face with a screw driver.

~~~~~
SOLUTION 2
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=1087732&SiteID=17

Integrated Windows Authentication was not turned on, on my Default SMTP Virtual Server.....WORKED FOR ME

 ~~~~

Adjusting Exchange 2003 mail flow settings for Exchange 2010

---

When bringing Exchange 2010 server into an existing Exchange 2003 environment, you can't initially send and receive Internet mail via the hub transport server. This is because Microsoft recommends that you place an edge transport server between the Internet and your back-end Exchange server.
An edge transport server is actually a hardened Exchange server that sits on the network perimeter. It maintains message hygiene as SMTP mail flows in and out of an Exchange organization. The edge transport server also shields back-end Exchange servers from direct Internet exposure.
Using an edge transport server is a good idea, but it's not a requirement. Given the current economic climate, I expect that a lot of organizations implementing Exchange 2010 will initially forgo the edge transport server to save money. If you decide to do this, you'll have to configure your hub transport server to send and receive Internet mail.
Note: If you decide not to use an edge transport server, I recommend that you place your mailbox server role on a different Exchange Server, if possible.
To prepare your hub transport server to send and receive Internet mail, create a send connector. The send connector allows the hub transport server to send mail directly to the Internet.
To create a send connector, follow these four steps:

1. Open the Exchange Management Console and navigate to Organization Configuration -> Hub Transport.
2. 
   
3. Go to the Actions pane and click on the New Send Connector link.
4. 
   
5. When the New Send Connector Wizard opens, set the connector's use to Internet.
6. 
   
7. Click Next and set the address to *.

The wizard's other options vary depending on your network configurations, but you do want to ensure that the source server option is set to use the Exchange 2010 hub transport server.
Exchange Server 2010 also uses a default receive connector to receive Internet mail. The hub transport server expects to receive mail from an edge transport server, not directly from the Internet. Because of this, the receive connector is configured to block all unauthenticated inbound SMTP traffic.
Since most Internet mail is not authenticated, you must configure the receive connector to allow anonymous SMTP connections. To do so:

1. Open the Exchange Management Console and navigate to Server Configuration -> Hub Transport Server.
2. 
   
3. Right-click on the receive connector and select Properties. Windows will display the receive connector's properties sheet.
4. 
   
5. Go to the Permission Groups tab and select the Anonymous Users check box.
6. 
   
7. Click OK.

Whether you use an edge transport server or a hub transport server, there is one final step to get mail flowing. You must redirect the inbound messages from one of your Exchange 2003 servers to either an edge transport server or a hub transport server.
Typically, the MX record for your domain will point to a firewall, which will reroute inbound SMTP traffic to an internal server. Therefore, you must reconfigure the firewall port forwarding to send SMTP traffic to the edge transport server or to the newly configured hub transport server.
Converting recipient policies to Exchange 2010 email address policies

Most Exchange organizations' internal domain names are different than the external domain names. For example, my primary external domain name is brienposey.com, but my Exchange servers reside on an internal domain named production.com. In this case, you must use recipient policies to define the appropriate external email addresses for your users.
Microsoft has replaced recipient policies with email address policies in Exchange Server 2007 and Exchange 2010. This means that when migrating from Exchange 2003, you'll need to convert your recipient policies into email address policies.
Doing so is quite simple. Open the Exchange Management Shell and enter the following command:
Get-EmailAddressPolicy | where {$_.RecipientFilterType –eq "Legacy"} | Set-EmailAddressPolicy –IncludeRecipients AllRecipients
This EMS command compiles a list of all mailboxes that use a legacy recipient policy. The command then converts the recipient policy into an email address policy.
http://searchexchange.techtarget.com/Adjusting-Exchange-2003-mail-flow-settings-for-Exchange-2010

RoutingGroup cmd

New-RoutingGroupConnector -Name "RGC 2003-2010" -SourceTransportServers "exchange2010FQDN" -TargetTransportServers "Exchange2003FQDN" -Cost 10 -Bidirectional $true -PublicFolderReferralsEnabled $true

SSL Enabling OWA 2003 using your own Certificate Authority

1. Configuring the CA 2. Creating Certificate Request 3. Getting the Pending Request accepted by our CA 4. Appending the Certificate to the Default Website 5. Enable SSL on the Default Website 6. Testing our SSL enabled Default Website

1. Configuring the Certificate Authority

The first thing to do is to decide which server should hold the Certicate Authority (CA) role, it could be any server as long as it’s at least a member server. If you have a single box setup, such as a Small Business Server (SBS), the decision shouldn’t be very hard.

Note:
In order to add the Certificate Service Web Enrollment component (subcomponent to CA), which we’re going to use in this article, the server needs to be running IIS, so if you haven’t already done so,  install IIS before continuing with this article. If you plan on installing the CA  component on the Exchange server itself, then there’s nothing to worry about, because as you know, Exchange 2003 relies heavily on IIS, which means It’s already installed.

To install the CA component, do the following:

• Click Start > Control Panel > Add or Remove Programs
• Select Add/Remove Windows Components
• Put a checkmark in Certificate Services

Below screen will popup as a warning, just click Yes > then Next

We now have to select what type of CA to use, choose Enterprise root CA and click Next

In the following screen we have to fill out the Common name for our CA, which in this article is mail.testdomain.com.
Leave the other fields untouched and click Next >

We now have the option of specifying an alternate location for the certificate database, database log, and configuration information. In this article we will use the defaults, which in most cases should be just fine.
Now click Next >

The Certificate Service component will be installed, when it’s completed, click Finish

2. Creating the Certificate Request

Now that we have installed the Certificate Services component, it’s time to create the Certificate Request for our Default Website. We should therefore do the following:

• Click Start > Administrative Tools > Internet Information Services (IIS) Manager
• Expand Websites > Right-click Default Website then select Properties
• Now hit the Directory Security tab
• Under Secure Communications click Server Certificate…

As we’re going to create a new certificate, leave the first option selected and click Next >

Because we’re using our own CA, select Prepare the request now, but send it later, then click Next >

Type a descriptive name for the Certificate and click Next >

We now need to enter our organization name and the organizational unit (which should be pretty self-explanatory),  then click Next >

In the next screen we need to pay extra attention, as the common name reflects the external FQDN (Fully Qualified Domain Name), to spell it out, this is the address external users have to type in their browsers in order to access OWA from the Internet.
Note: As many (especially small to midsized) companies don’t publish their Exchange servers directly to the Internet, but instead runs the Exchange server on a private IP address, they let their ISP’s handle their external DNS settings. In most cases the ISP creates a so called A record named mail.domain.com pointing to the  company’s public IP address, which then forwards the appropriate port (443) to the Exchange servers internal IP address.

When your have entered a Common Name click Next >

Now it’s time to specify the Country/Region, State/Province and City/locality, this shouldn’t need any further explanation, when you have filled out each field, click Next >

In the below screen we have to enter the name of the certificate request we’re creating, the default is just fine, click Next >

In this screen we can see all the information we filled in during the previous IIS Certificate Wizard screens, if you should have made a mistake, this is your last chance to correct it. If everything looks fine click Next >

And finally we can click Finish.

3. Getting the Pending Request accepted by our Certificate Authority

Now that we have a pending Certificate Request, we need to have it accepted by our CA, which is done the following way:

• On the server open Internet Explorer
• Type http://server/certsrv

Note: In order to access the Certsvr virtual folder, you may be prompted to enter a  valid username/password, if this is the case use the Administrator account. When you have been validated the Windows 2003 Server will most probably block the content of the CertSrv virtual folder, which means you wil  have  to add it to your trusted sites in order to continue.

Now that you’re welcomed by the Certificate Services, select Request a Certificate

Click advanced certificate request

Under Advanced Certificate Request click Submit a certificate request by using a base-64-encoded CMC or PKCS #10 file, or submit a renewal request by using a base-64-encoded PKCS #7 file

Now we need to insert the content of the certreq.txt file we created earlier, you can do this by clicking the Browse for a file to insert or by opening the certreq.txt file in notepad, then copy/paste the content as shown in the screen below, then click Submit >

Now select Base 64 encoded then click Download certificate

Click Save

Choose to save the certnew.cer on the C: drive > then click Save

Close the Microsoft Certificate Services IE window.

4. Appending the Certificate to the Default Website

Okay it’s time to append the approved Certificate to our Default Website, to accomplish this we need to do the following:

• Click Start > Administrative Tools > Internet Information Services (IIS) Manager
• Expand Websites > Right-click Default Website then select Properties
• Now select the Directory Security tab
• Under Secure Communications click Server Certificate… > then Next

Select Process the pending request and install the certificate > click Next >

Unless you have any specific requirements to what port SSL should run at, leave the default (443) untouched, then click Next >

You will now see a summary of the Certificate, again if you should have made any mistakes during the previous wizard screens, this is the final chance to correct them, otherwise just click Next >

The Certificate has now been successfully installed and you can click Finish

5. Enabling SSL on the Default Website

We have now appended the Certificate to our Default Website, but before the data transmitted between the clients and the server is encrypted, we need to click the Edit… button under Secure Communications.

Here we should put a checkmark in Require Secure Channel (SSL) and Require 128-bit encryption just like below:

Now click OK.

6. Testing our SSL enabled Default Website

Now that we have gone through all the configuration steps necessary to enable SSL on our Default Website, it’s time to test if our configuration actually works.
From the server (or a client) open Internet Explorer, then type:
http://exchange_server/exchange
You should get a screen similar to the one shown below:

This is absolutely fine, as we shouldn’t be allowed to access the Default Website (and any virtual folders below) through an unsecure connection. Instead we should make a secure connetion which is done by typing https, therefore type below URL instead:
https://exchange_server/exchange
The following box should appear:

Note: You may have noticed the yellow warning sign, this informs us The name on the security certificate is invalid or does not match the name of the site. Don’t worry there’s nothing wrong with this, the reason why it appears is because we aren’t accessing OWA through the common name, which we specified when the certificate was created. When you access OWA from an external client through mail.testdomain.com/exchange, this warning will disappear.
Click Yes
You will now be prompted for a valid username/password in order to enter your mailbox, for testing purposes just use the administrator account, like shown below:

Now click OK
We should now see the Administrator mailbox.

Notice the yellow padlock in the lower right corner, a locked padlock indicates a secure connection, which means OWA now uses SSL.

Final words

Even though it’s possible to run your OWA environments without securing it with a SSL certificate, I strongly advise against doing so, as this would mean any traffic send between the external OWA clients, and the Exchange server would be sent in cleartext (this includes the authentication process). As you now know SSL provides us with 128-bit encryption, but be aware enabling SSL in your OWA  environment isn’t an optimal security solution, in addition to enabling SSL, you should at least have some kind of firewall (such as an ISA server) placed in front of your Exchange server(s). You might also consider enabling the new Exchange 2003 functionality Forms Based Authentication, which provides a few additional benefits such as a new logon screen, which, among other things, uses session cookies to make the OWA sessions more secure, unfortunately the Forms Based Authentication functionality is out of the scope of this article, but I will at some point of time in the near future write another article covering this funtionality.
That was it for this time, I hope you enjoyed the article.
http://www.msexchange.org/tutorials/SSL_Enabling_OWA_2003.html

Exchange Server stores information in a database and uses log files for transactional processing. To restore, defragment  or repair a database, the ESEUTIL tool is essential . It is always possible to recover data when the database is lost, if you have backed up the database. 

Exchange Server Log File Replay

In my previous article, Exchange Database Technologies, I discussed the underlying database technology in Exchange Server, the Extensible Storage Engine or ESE. One of the most important points in that article was that all changes to the Exchange Server database go through the log files. This is done for recovery purposes. Let’s look at the log files, and the replay of log files in case of a recovery scenario…

Creation of the database

After you create a Storage Group in the Exchange Management Console, we have an empty directory on disk. The only thing that actually happens is setting a property for the Storage Group in Active Directory, so no log files have been created yet.
When a database is created in the Exchange Management Console there is still an empty directory on the disk. Again, the only thing that happened is setting a property in Active Directory.
When the database is actually mounted, these things will happen on disk:

• Active Directory is checked for the location of the log files;
• When no log files are found a new set of log files is created with an lGeneration number of 1;
• Active Directory is checked for the location of the database;
• Since this is an initial mounting a new database is created;
• At this point log files, a database file and a checkpoint file have been created and the Exchange Server’s database is ready to use.

Tip.
In the old Backoffice 4.x resource kit there was a small utility called “mailstorm”. This MAPI utility was very useful to send a massive amount of messages in a short timeframe. Using mailstorm it is possible to see the creation of log files. You can still find the original mailstorm utility here:. Unfortunately mailstorm doesn’t work with Exchange Server 2007 anymore, but there’s a PowerShell script available having the same functionality. Using this script a variety of of predefined messages can be sent to your Exchange Server 2007 to check the log file creation. The PowerShell version of mailstorm can be downloaded here:

Create a user account in Active Directory and create a mailbox using the Exchange Management Console. Log on to the mailbox and start sending messages until you have a couple of messages and a couple of log files.
What happens when the database file is lost? If we have all the log files still available it should be possible to retrieve all information. Remember what I wrote in my previous article: “everything is logged in the log files, even the creation of database files!”
If you dismount the database, delete the database file “mailbox database.edb” or whatever name you have given the database and try to mount the database again, the following error is raised:

<
Figure  1. Error message when a mailbox database appears to be missing

In my humble opinion, the yellow exclamation mark should be replaced with a very large red “X” since this is a very important message. When you click “Yes” a new mailbox will be created in the same location as the “old” database. This is a completely new database. Although it has the same name “mailbox database.edb” as in the previous step, it has new signatures, as explained in my previous article, which Exchange Server uses to link databases and log files file together. Recovery of old log files will not result in information being replayed into the database because it is another database in this scenario. And remember, since all information is logged into the log file the creation of this new database is also logged. Choose No, and then delete the checkpoint file E00.chk and try to mount the database again. No error message is raised and the database is mounted. Even better, when you log on to your test mailbox you will see that no information is lost either!
This is what happens when you do click “Yes”: during the mount process Exchange Server cannot find the database file and it cannot find the checkpoint file. Therefore it starts to recover all information by replaying the available log files.  It starts with the oldest log file E0000000001.log which also contains the creation of the initial database file. All information in the other log files is replayed into this database until it reaches the end of the last log file E00.log. The database is mounted and ready to use.
When Exchange Server cannot find the database file but it does find the checkpoint file it will not replay the log files. It starts at the end of the last log file E00.log and it will create a new database.
How can you tell which files belong together?
Dismount the Exchange Server’s database, open a command prompt in the database’s directory and check the header information using the ESEUTIL tool. You will find the following information:

K:\sg1>eseutil /mh "mailbox database.edb"

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating FILE DUMP mode...
         Database: mailbox database.edb

     DB Signature: Create time:08/25/2008 22:54:52 Rand:4416518 Computer:

    Log Signature: Create time:08/25/2008 22:54:50 Rand:4412688 Computer:

Operation completed successfully in 0.140 seconds.


K:\sg1>

When you check the header information of the log files with the ESEUTIL tool you will find corresponding information:

L:\sg1>eseutil /ml e000000001a.log

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

      Signature: Create time:08/25/2008 22:54:50 Rand:4412688 Computer:

      1 k:\sg1\Mailbox Database.edb
                 Signature: Create time:08/25/2008 22:54:52 Rand:4416518 Computer:

Operation completed successfully in 0.60 seconds.

L:\sg1>

Note: both screen outputs have been edited for readability

As you can see both the log file signature and the database signature match, so these files belong together. When you (accidentally) create a new mailbox you will find other information in the database header:

L:\sg1>eseutil /ml e000000001c.log

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

      Signature: Create time:08/25/2008 22:54:50 Rand:4412688 Computer:

      1 k:\sg1\Mailbox Database.edb
                 Signature: Create time:09/02/2008 07:27:28 Rand:963593 Computer
:
Operation completed successfully in 0.70 seconds.

L:\sg1>

As you can see the log signature hasn’t changed (still the same set of log files) but the database has a new signature, meaning that although the database has the same name and it is in the same location it is a new database!

Key take-a-way: the checkpoint file determines if log files are replayed and where log file replay will start and therefore what happens during the mounting process.

Offline backups

You can create backups by copying the database files to a safe location. The steps to do so are:

1. Dismount the database (meaning it is not available!)
2. Copy the database file to a safe location
3. Mount the database
4. Perform a consistency check on the database copy
5. If everything is ok, delete the log files

The first three steps do not need any further explanation. But what log files can you safely delete?
When dismounting the database all information in the log files that is not yet committed to the database is flushed to the database file. When all data is flushed the files are closed.
You can check in the database header information when the database was dismounted by looking at the “last detach” information. Also check that the database is in a clean shutdown and does not need any log files for mounting.

K:\sg1>eseutil /mh "mailbox database.edb"

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

            State: Clean Shutdown
     Log Required: 0-0 (0x0-0x0)
    Log Committed: 0-0 (0x0-0x0)

      Last Detach: (0x11,27,35)  09/02/2008 16:34:34

Operation completed successfully in 0.150 seconds.

K:\sg1>

So in this specific scenario, all log file older than E0000000011.log can safely be deleted. All information in these log files is flushed to the database. Why not log file E0000000011.log itself? There can be information logged in the log file in this same log file beyond this point after mounting the database again.
The offline copy of the database has also been checked for consistency. A database can contain corrupt pages, and as long as these pages are not read by the Exchange server you will never know these pages are corrupt. Suppose somebody is on a one year sabbatical leave and his mailbox data is never accessed, corrupt pages can exists for this period of time without being noticed.
So you have to check the offline copy for any inconsistencies using the ESEUTIL tool with the /K option:

K:\backup>eseutil /k "mailbox database.edb"
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating CHECKSUM mode...
        Database: mailbox database.edb
  Temp. Database: TEMPCHKSUM5592.EDB

File: mailbox database.edb

                     Checksum Status (% complete)e)

          0    10   20   30   40   50   60   70   80   90  1000000
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................


1282 pages seen
0 bad checksums
0 correctable checksums
388 uninitialized pages
0 wrong page numbers
0x7dd5 highest dbtime (pgno 0x193)

161 reads performed
10 MB read
1 seconds taken
10 MB/second
17350 milliseconds used
107 milliseconds per read
280 milliseconds for the slowest read
0 milliseconds for the fastest read

Operation completed successfully in 0.471 seconds.
K:\backup>

When using the /K option all pages in the database are read and their checksum information is read and checked. When everything is ok, you can safely continue.

Note. Starting with Exchange Server 2003 Service Pack 1 Exchange has error correcting code for checksum errors. If a certain page contains a checksum error, which is usually caused by only one bit being “flipped”, Exchange Server can automatically correct this bit. When the page is flushed to disk the error is automatically corrected. Please check the Microsoft kb article 867626 (Microsoft kb article 867626) for more background information regarding this change.

 Offline Restore

When something happens we have to restore the offline backup on the Exchange server. An offline restore requires the following steps.

1. Copy the offline backup mailbox file to the correct location<
2. Replay the log files
3. Mount the database

Now remember the first part of this article. If we leave the checkpoint file in its original location the Exchange Server will check this file and will determine that no replay is needed and the copy of the database will be mounted.
If we delete the checkpoint file the Exchange Server will start replaying log files from the oldest log file available. This will result in all information after the offline backup being replayed into the copy of the database. The result will be a complete up-to-date database.
If you check the event log of the Exchange server you can see which log files were replayed. For every log file an entry is logged with ID 301 from the ESE Source.

Figure 2. Log file replay events in the event viewer

Besides just mounting the database it is also possible to replay the log files into the database manually using the ESEUTIL tool. This can be achieved using the /R option for Recovery.

L:\sg1>eseutil /R E00 /i /dK:\SG1

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating RECOVERY mode...
    Logfile base name: E00
            Log files: t;t;
         System files: t;t;
   Database Directory: K:\SG1

Performing soft recovery...
                      Restore Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................

Operation completed successfully in 5.819 seconds.

L:\sg1>

Note. the drive letter K: immediately follows the /d option, there’s no space in between.

Online Maintenance

There’s an internal process in the Exchange server that’s responsible for the maintenance of the mailbox database. This is called the online maintenance, or OLM. OLM runs every night, by default between 1 am and 5 am. Online maintenance consists of 10 tasks that are performed during this period. Five of them are specifically for the Public Folder database and five of them are for the Public Folder database and the mailbox database. I want to highlight two of them in this article:
OLM checks the mailbox database for deleted messages that are beyond their retention time. If there are, they are fully deleted from the database and their pages are freed up (that is, marked as “white space”). OLM is also responsible for permanently deleting mailboxes that are beyond their retention time. If messages and mailboxes are beyond their retention time they are really gone, the only way to get these back is by restoring an old backup of the mailbox database. Archiving solutions can help here, but they are out of the scope of this article. When the mailboxes and messages are permanently deleted the pages they used are freed up and available for new messages. But the free pages are scattered across the database. The OLM also performs an online defragmentation of the database file by consolidating freed up pages from the previous step and if possible, placing them in one contiguous block of free data. This makes the addition of new data to the database more efficient since the data doesn’t have to be split and separated across multiple segments of the database.
Online Maintenance is a very disk intensive application and it should be monitored very closely. If the Exchange server cannot finish the OLM in the designated timeframe errors are logged in the event log. Carefully examine these errors and check what causes them.
Make sure that no other processes are working with the mailbox database at the same time. Typically maintenance processes are scheduled at night, such as backups or archiving. I will cover online backups in the next article in this series. The movement of large numbers of mailboxes, which is also often scheduled during the night, will also have a negative impact on the OLM.
For a complete overview of the online maintenance please visit the Microsoft Exchange team blog

Key take-a-way: Online Maintenance is responsible for online defragmentation; it defragments inside the database only. This does not result in a decrease of the physical size of the mailbox database.

 Offline defragmentation

Please note that offline defragmentation should never be considered a part of normal maintenance of an Exchange database. Only if your free space exceeds 30% of your total database size, or if you are told to do a defragmentation by Microsoft Customer Support Services, should you consider doing an offline defragmentation.
While the online maintenance does defragmentation within in the database, it does not compact the database in physical size. If you have a 100 GB database and it has only 10 GB of data in it, the online maintenance will eventually free up 90 GB within the database (it will be marked as “white space” or “available space” within the logical tables of the database). The file “mailbox database.edb” as it resides on disk still remains 100 GB. To compact the database we have to perform an offline defragmentation. This can only be done using the ESEUTIL tool with the /d option. As the name implies, this has to be done when the database is offline.

K:\sg1>eseutil /d "mailbox database.edb"

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating DEFRAGMENTATION mode...
            Database: mailbox database.edb

                  Defragmentation Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100000000
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................


Moving 'TEMPDFRG2256.EDB' to 'Mailbox Database.edb'... DONE!

Note:
  It is recommended that you immediately perform a full backup
  of this database. If you restore a backup made before the
  defragmentation, the database will be rolled back to the state
  it was in at the time of that backup.

Operation completed successfully in 8.913 seconds.

K:\sg1>

If you look closely at this output you can already see what happens during an offline defragmentation. A new mailbox database file is created is with a random name, in this case TEMPDFRG2256.EDB. The data from the original mailbox database is copied to this temporary database. Of course existing white space in the original database is not copied to the new database. When finished copying the temporary file is renamed to the original file and is now called “mailbox database.edb”. So this is a new file, containing only the database that resided in the original file. In the previous example with the 100 GB database with only 10 GB of data, when an offline defragmentation is performed we will be left with a new 10 GB database. There is some overhead in this, so the sizing can vary +/- 10%.
You must realise that this is a new database with new signatures etc. so there is no possibility of recovering theto recover data in existing log files into this new database. This is the reason you should make a new backup immediately after performing an offline defragmentation.
Please note that if you do not redirect the temporary file created by Offline Defragmentation, the disk volume containing the Exchange mailbox database should have at least 110% of the size of the mailbox database for available disk space.
In the Exchange Server 5.5 timeframe it was a best practice to perform an offline defragmentation every months. This would create a new database with new tables, indices etc. Microsoft has spent a tremendous amount of development in the database engine. Nowadays the database engine is that good and stable that an offline defrag has only to be performed after a repair action, or when large amounts of data can be freed up. This can be the case after the deletion of a large number of mailboxes or after moving a lot of mailboxes to another database.

Worst case scenario: Repairing your database

The worst thing that can happen when running Exchange is that your server crashes and your database refuses to mount. After checking if the Information Store is running and maybe rebooting the server it still won’t mount and the following error is raised:

  Figure 3 - Error message when the database won't mount after a crash

In this example one of the log files needed for the mounting process is not available anymore (I deleted one of these log files for demonstration). If this occurs it can be checked by comparing the values in the “logs needed” section of the database header against the actual log files that are on disk.
So now we have a database that is not consistent anymore and that has to be fixed before it can be used again. To accomplish the ESEUTIL tool should be used again but now with the /P option (for repair).
Note. Only perform this step when you have made a copy of all databases, log files and checkpoint file and placed them in a safe location. Performing a repair action can result in data loss!
When you enter this command an error is raised stating that if you continue information might be lost. What actually happens is that ESEUTIL checks all information in the database and it checks all tables and pointers. If pointers do not point to the correct location, or information in this location cannot be read, the pointer is deleted and the index entry is cleaned up. All information that the pointer points to will be lost!

K:\sg1>eseutil /p "mailbox database.edb"

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating REPAIR mode...
        Database: mailbox database.edb
  Temp. Database: TEMPREPAIR1888.EDB

Checking database integrity.

The database is not up-to-date. This operation may find that
this database is corrupt because data from the log files has
yet to be placed in the database.

To ensure the database is up-to-date please use the 'Recovery' operation.


                     Scanning Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................


Integrity check successful.

Note:
  It is recommended that you immediately perform a full backup
  of this database. If you restore a backup made before the
  repair, the database will be rolled back to the state
  it was in at the time of that backup.

Operation completed successfully in 150.747 seconds.


K:\sg1>

At the end of this operation you will have a database that is in a consistent state, but indexes in the database might be non-contiguous and thus less efficient..As a Microsoft best-practice an offline defragmentation should be performed after a database repair.  This will create a new database, with the existing data but with new tables, indices etc. Both ESEUTIL /P and ESEUTIL /D will fix issues on a low (database) level but not on an application level. After performing a repair and on offline defragmentation on your database you have to run the ISINTEG tool. This will check the database on an application (=logical) level and if issues are found fix them. When completely finished do not forget to backup your database immediately.

Note. Perform these steps only if you have backed up the database(s), log files and the checkpoint file since performing a repair action can lead to data loss!

To be 100% sure that all issues are fixed on all levels you can create a new mailbox database and move all mailboxes from the repaired database to the new database. When all mailboxes are moved you can delete the repaired database. But please be aware that a repair action with ESEUTIL can delete data, so despite all activities there might be circumstances that end users mail can be lost. Unfortunately it is not predictable what and how much will be lost.
You can find more information on the Microsoft website: http://technet.microsoft.com/en-us/library/aa997152(EXCHG.80).aspx and http://msexchangeteam.com/archive/2004/06/18/159413.aspx

Note. Of course it is also possible to restore a backup of the database and recover the existing log files up to the point of the missing log files. If possible this is most likely the best and fastest option you will have.

When using ESEUTIL, there is one very important issue. ESEUTIL is very powerful, but not very fast. Depending on the hardware in use for the Exchange server, as a rule of thumb ESEUTIL will process the Exchange database with a rate of between 5 GB/hour and 10 GB/hour. When you have a 250 GB Exchange database, a repair action on your database would take 25 hours or longer to complete A following offline defragmentation can also take up to 25 hours, so you will see an outage of at least 50 hours. I don’t take the ISINTEG tool and moving the mailboxes to a complete new database into account, but I’m pretty confident that this will not match your Service Level Agreement (SLA). On an Exchange server using a single copy database, i.e. a single server or a single copy cluster, it is best to use a maximum database size of 50 GB. This will keep your database in a manageable state from a service level point of view.
So why does Microsoft suggest using a maximum 200 GB database size in an Exchange Server 2007 Continuous Cluster Replication  (CCR) environment? As explained in my first article, you have an extra copy of your database available in your Exchange 2007 cluster. So when the database of the active node crashes and is beyond recovery (i.e. cannot be mounted anymore) the passive copy will take over and start servicing your (Outlook) clients. When this happens you have time to start investigating the broken node of your cluster and repair it. You are not tied to the 5 GB/hour processing time of ESEUTIL, which ought to make you more comfortable with your Service Level Agreement.

Conclusion

Exchange Server stores information in a database and uses log files for transactional processing. This way it is always possible to recover data when the database is lost, if you have backed up the database. The tool for database maintenance is ESEUTIL. Using ESEUTIL you can check your database, recover data from the log files and replay it into your database or repair your database when it’s corrupt.
When you need to compact your database, for example after you deleted or moved multiple mailboxes, you have to perform an offline defragmentation, but remember to create a new backup after performing an offline defragmentation.
One important thing to remember is the checkpoint file. Although a tiny file only 8KB in size it has a major impact on the results when either recovering data using ESEUTIL or mounting the database in the Exchange Management Console.
When you run into issues in your test environment that isn’t a problem. You can and you should start working with database and ESETUIL to see what exactly happens in the different scenarios. Make sure you fully understand (and document!) all the necessary steps for a successful recovery when you lose your database. However, do this in your TEST environment, not in your live/production environment.
When problems happen in your production environment and you face a major outage, ask Microsoft Customer Support for help. Microsoft CSS will guide you through a successful recovery, but when you have some experience in your test environment you know at least what’s happening and why Microsoft Customer Support asks you to perform the various steps.
In my last article I will talk about online backups and VSS backups and explain the differences between these and the steps documented in this article.

http://www.simple-talk.com/content/article.aspx?article=572

Written by by Jaap Wesselius
22 September 2008

Eseutil /r - Error 528 - Exchange DB recovery

While try to recover priv1.edb and pub1.edb

c:\\Program Files\Exchsrvr\MDBDATA>"c:\\Program Files\Exchsrvr\bin\Eseutil" /r e00

Initiating RECOVERY mode...
Logfile base name: e00
Log files:
System files:

Performing soft recovery...


Operation terminated with error -528 Current log file missing> after xxx seconds

Cause:
Missing valid Exx.log files

Solution:
1. Look folder c:\\Program Files\Exchsrvr\MDBDATA2. Copy last E000xxx.log, E00.log and E00.chk to other place
3. Delete E00.log
4. Rename last E000xxx.log to E00.log
5. Execute
c:\\Program Files\Exchsrvr\MDBDATA>"c:\\Program Files\Exchsrvr\bin\Eseutil" /r e00
5. You may need this procedures with following order:
c:\\Program Files\Exchsrvr\MDBDATA>"c:\\Program Files\Exchsrvr\bin\Eseutil" /g
c:\\Program Files\Exchsrvr\MDBDATA>"c:\\Program Files\Exchsrvr\bin\Eseutil" /p
c:\\Program Files\Exchsrvr\MDBDATA>"c:\\Program Files\Exchsrvr\bin\Eseutil" /d 


Copy from http://iwan-it-admin-tips.blogspot.com/2009/01/eseutil-error-528.html