Ecubed, Day 1: A word about write caching and Exchange databases

I’m spending this week at the Exchange Emergent Evolution (Ecubed) conference in downtown Seattle. It’s a small, intimate gathering for technically minded Exchange professionals to get together and get the latest and greatest advanced training. The big downer is that there’s no wireless connectivity, so I can’t blog from the conference. On the other hand, the office can’t e-mail me, so I think I’ll live.

For Monday and Tuesday, I am in the Storage Best Practices session taught by Microsoft’s Dave Lalor; for the rest of the week, I’ll be in the Defense in Depth: Exchange Security session. The first day of the storage session has been an absolute blast and joy; Dave really knows his subject, communicates his excitement and knowledge very well, and has access to the live performance data from the production Exchange servers at Microsoft to illustrate his lessons. It’s awfully hard to argue with someone who is walking through some incredible graphs of high-load production Exchange 2003 SP1 servers purring along like kittens.

Some tasty tidbits from day one (if I go by too quickly, don’t worry; I plan to write these up in more detail later):

  • Remember the old advice to always disable write caching on your controllers? This advice comes from the days when clusters used shared SCSI disk systems; each host had its own indepedent SCSI controller on a shared SCSI bus. If a host failed, any data in the cache that hadn’t been committed to disk was lost, creating a hole in your Exchange databases that would nuke them from orbit. If you’re the last person still using this “ghetto clustering” (as one of my fellow conference goers dubbed it), keep following this advice. If you’re using a modern SAN solution, enable your write caching and enable it now. The cache is not on the host, but on the controllers in the array, and the better arrays with redundant controllers make sure cache information is shared between all controllers (verify that yours does this, of course!) The resulting performance win will make your Exchange servers much happier.
  • If you have a dedicated array for your Exchange boxes, don’t bother with read caching on the array. Exchange does not benefit at all from read caching.
  • Contrary to popular opinion, RAID 5 volumes may in fact be suitable for your Exchange database mounts depending on the particlar I/O characteristics of your traffic. A big part of this is how your RAID controller handles write caching. Any type of RAID array introduces additional I/O operations (IOPs) behind the controller; a RAID 10 array (a stripe of mirrored disk pairs) turns every one write IOP at the host into two write IOPs at the controller. RAID 5 arrays can incur an additional penalty; this varies by vendor and model. A common penalty factor is 4: each write IOP at the host incurs an additional two read IOPs and two write IOPs at the controller. With detailed knowledge of your disk performance, your database design, the particular of your RAID controller, and some careful attention to detail, you can produce more than acceptable performance for Exchange databases on a RAID 5 volume.

More later!