Nutanix has recently published a Best Practice Guide for Mictosoft Exchange on VMware vSphere and Josh Odgers explains some of it’s contents  and benefits of Exchange on Nutanix in his blog article here. If you are interested in virtualizing Exchange, and/or using Nutanix, you might want to get hold of the guide and have a read through it. It explains how to simply set up Exchange on Nutanix, the benefits of it, how it compares to a traditional physical JBOD approach and much more. The paper introduces the capability of running Exchange on the Nutanix NX-8150 nodes, which have been specially designed to run large applications, such as Exchange, SQL Server, Oracle and SAP. This is the node type Josh Odgers and I used as part of a design capable of hosting 1.4 million Exchange 2013 Mailboxes, which demonstrates the building block architecture of Nutanix and the ability to scale to meet requirements for large environments. Let’s take a look at that design at a high level.

This design was created as part of a response to an RFI (Request For Information) for a customer in Europe. They wanted to support 1.4 million Exchange 2013 mailboxes at 500MB each with 50 messages per day. The number of messages per day and the size of mailboxes can be adjusted by adding more nodes to the design.

The Nutanix NX-8150 nodes  (1 node per 2RU) used in the below example are configured with 2 x E5-2690 v2 sockets (10 cores each @ 3GHz) and 256GB RAM. They each have 10TB usable capacity per node before any dedupe or compression benefits, and can be configured with up to 6.4TB of raw flash capacity per node (3.2TB raw flash per node used in this example). Note: Diagrams are not exactly to scale as there are additional RU remaining unused per rack.

Assumptions:

1. 500MB Mailbox for 1.4 Million users

2. 50 Messages / Day

3. Following MS recommendations as follows:

a. 10% Hypervisor Overhead

b. 80% CPU utilization per server maximum

4. No Capacity savings from Compression or deduplication taken into consideration

5. Site Resiliency is mandatory

Configuration

1. 16 x 9 node DAGs with mixed server roles (MBX+CAS server roles combined) per VM with 24 vCPUs (PreferHT) and 96GB RAM

2. Total of 144 x NX-8150 w/ 256GB RAM inc 32GB RAM for CVM plus 128GB remaining to support MBX server vMotion for maintenance (per site)

3. Secondary Datacenter also requires 144 x NX-8150 nodes to host 1 DAG and 1 LAG copy

4. 512MB mailboxes @ 100% capacity requires 6939GB per node which leaves >2TB free per node in the worst case scenario

5. Mailbox Servers will run at <80% utilization (at both sites) as per MS recommendations

6. Exchange Transactional IOPS Per Node required is 474 – Note: Jetstress results in N+1 is over 600 IOPS/Node

7. 1 x Active Directory Global Catalog Server per node with 2 vCPUs / 4GB RAM (or equiv number of cores in larger AD VMs) Note: 313 GC Cores required per site

8. A total of 10 racks required per site (20 racks in total). The entire Nutanix environment can be managed across both data centers from a single management console (PRISM Central).

Recommendation

1. Nutanix/vSphere cluster sizes of 9 nodes striped across racks, 16 nodes per rack (each a member of a different Nutanix cluster) – Note: 16 x NX-8150 is 32RU leaving 10RU – 2 x 1RU ToR Switches , 2 x 1RU Cable Management, 6RU remaining (additional 1RU in each of the network spine racks required)

2. Mailbox Server placement being 1 DAG member per Nutanix Cluster i.e. extremely high resiliency as 1 cluster failure does not bring down a single DAG

3. Use 2 x 48 port 10GbE MoR (Middle of Rack) switches per 2 Rack pair and 2 x 48 Port 1GbE switches per rack pair (one switch per rack with servers cross connected) – 32 ports per 10GbE switch for mailbox servers and 16 per 1GbE switch for IPMI (Out of Band Management), optionally could also connect servers with 1GbE in addition to IPMI

4. Start with 8 nodes per DAG and 4 dags (32 nodes per site) for a total of 64 Nodes across both active/active sites (~300K mailboxes @ ~4688 per node), grow by additional 64 nodes, then 16 nodes (~75K mailboxes) at a time until final configuration is met.

Initial Exchange Infrastructure Layout (~300K Mailboxes)

Next Step Exchange Infrastructure Layout (~600K Mailboxes)

Third Step Exchange Infrastructure Layout (~675K Mailboxes)

Fourth Step Exchange Infrastructure Layout (~750K Mailboxes)

Subsequent Steps for Exchange Infrastructure Layout (Adding ~75K mailboxes per increment)

Final Exchange Infrastructure Layout (1.4M Mailboxes)

Exchange Network Infrastructure Layout

Final Word

It is easy with Nutanix to start much smaller and design exchange environments for much lower number of mailboxes (or much larger mailbox size) and for smaller growth increments (<500 – > 2000 mailboxes). This example was for a very large environment and specific customer requirements, and shows for this large environment a small(ish) starting point and growth increments, until it reaches it’s final size. The diagrams include all compute, storage and networking equipment required to host the specified number of mailboxes. At the time of publishing this article the outcome of the RFI that this design was created for is unknown. If Nutanix is successful and the customer agrees to be a reference you’ll be sure to hear about it. The purpose of this article was to show at a high level how you could use Nutanix NX-8150’s and a building block approach to design a large scale environment suitable for Exchange. Although the target for this design, and the Exchange Best Practices paper is VMware vSphere you could also use Microsoft Hyper-V (Nutanix Exchange Best Practice Guide on Microsoft Hyper-V is in the works).

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – 2014 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.