Home > Business Critical Applications, VMware > The Status of Microsoft Failover Clustering Support on VMware vSphere 5.1

The Status of Microsoft Failover Clustering Support on VMware vSphere 5.1

March 22, 2013

The number of enquiries I’ve been receiving regarding Microsoft Failover Clustering, especially for Microsoft SQL Server Databases has skyrocketed in the past few weeks. I have been receiving a number of enquiries from customers and also from partners including cloud service providers. As a result I thought I’d write this article to help you understand what the current status is of support for Microsoft Failover Clustering on VMware vSphere 5.1 (GA) and with regard to some VMware products.

Background Reading

Firstly there are two main VMware knowledge base article that outline the support statements of Microsoft Failover Clustering and Microsoft Cluster Services on VMware vSphere. They are as follows:

Microsoft Cluster Service (MSCS) support on ESXi/ESX (1004617)

Microsoft Clustering on VMware vSphere: Guidelines for Supported Configurations (1037959)

This article only applies to vSphere 5.1. The rule book has been rewritten with vSphere 5.5, check out my article on vSphere 5.5 Windows Failover Clustering Support.

Clustering and VMware Solutions

In addition to the above there are specific mention of clustering configurations for the VMware technologies that support it, such as for the vCloud Director SQL Database, which was introduced in vCD 5.1 and covered in my article Clustering Support on vCloud Director and vCenter Databases. The golden rule is this. If VMware does not specifically document a clustering solution as being supported then it is NOT supported. vCenter Server from version 4.0 to current 5.1 GA does not support a clustered Database, be it Oracle RAC or SQL Server. It has not been tested by VMware and is therefore not supported. This may well change in the future as VMware recognises the need to provide alternative high availability solutions for the vCenter Database and I will update this article accordingly. However currently the supported high availability solutions for the vCenter and its database are VMware HA, and vCenter Server Heartbeat. Clustering of the vCenter Server itself is also not supported by VMware but is covered by KB article 1024051 - Supported vCenter Server high availability options.

Customers with production support who wish to run Oracle RAC for the DB for vCenter (not SSO, as that doesn’t work) can get support from the VMware Oracle Support Team under VMware’s Expanded Oracle Support Policy. But they will be limited by the capabilities of vCenter itself, if any. I do know a number of customers running vCenter DB (Not SSO) on Oracle RAC in an active/passive service configuration and it has been fine for years. Also I expect the official support statement to change in the future as the testing for vCenter and RAC is completed.

Not supported does not always mean something doesn’t work. But it does mean it hasn’t been tested by VMware and therefore VMware can’t stand behind the configuration as a supported solution. If it’s not documented as supported, then it’s not supported.

The Status of Microsoft Failover Clustering Support on VMware vSphere 5.1

VMware has done a lot of work to enhance support for Microsoft Failover Clustering and its predecessor Microsoft Cluster Services on VMware vSphere 5.1 to support larger cluster sizes. You can now support up to 5 nodes in a virtual Microsoft Failover Cluster on vSphere 5.1. This is great news for environments where two nodes was not enough, even when combined with the additional availability of VMware HA. I’ve implemented a number of solutions where Microsoft Failover Clustering was used successfully in the cases where it was justified and within the limits that were supported. Strong justification and support constraints are two things I’d like you to think about as you read further.

You can still do hybrid Physical and Virtual clusters, and you can also still do cluster-in-a-box with VMDK’s (dev / test of cluster functionality itself not for high availability). VMware Site Recovery Manager is also supported to protect Microsoft Failover Clusters from a DR perspective and there are a number of different configurations you can use such as multi-node to single node, or multi-node to multi-node. This really does make DR for the cluster easy, less error prone, and of the recovery plan itself once it is initiated is automated and provides audit reporting. VMware HA is fully supported, however VMware recommends you implement anti-affinity rules to ensure cluster nodes are prevented from start up on the same physical host.

So what are the gotcha’s or caveats I hear you ask? Well there are a few gaps in support that you should be aware of when developing your solution architecture. I’ll also cover some of the other valid options you have for high availability later as well and some of the impacts of using Microsoft Failover Clustering. This list is in no particular order.

  1. Clustering Across Boxes (i.e. traditional clustering for high availability purposes) is not supported with the use of VMDK’s or Virtual Mode RDM (vRDM). You must use Physical Mode RDM’s (pRDM) due to the requirement of persistent SCSI reservations.
  2. Due to the requirement to use pRDM’s there is no support for doing backups with vSphere API’s for Data Protection (vADP). So you must use in guest agents for backup.
  3. The is no support for vMotion or DRS with Microsoft Failover Clusters as they use shared disks and a shared SCSI bus. Any attempt to migrate a cluster node will be met with an error message.  This doesn’t mean you can’t deploy a Microsoft Failover Cluster inside a VMware DRS cluster, because you can and it’s fine, it just means that DRS can’t automatically migrate the Microsoft Failover Cluster nodes automatically because vMotion isn’t supported.
  4. Windows Server 2012 Failover Clustering is not supported currently. Period. Not even with in-guest iSCSI. [Updated 21/06/2013] Except with non-shared disk access, in-guest iSCSI, or in-guest SMB storage access. MS SQL Server 2012 on top of Windows Server 2012 with AlwaysOn Availability Groups is supported as it does not require shared disk. See the VMware KB Microsoft Clustering on VMware vSphere: Guidelines for Supported Configurations (1037959) and my article Windows Server 2012 Failover Clustering Now Supported By VMware With Some Caveats.
  5. There is no support for Native iSCSI (where an RDM is presented via the host iSCSI initiator or iSCSI HBA to a guest)
  6. There is no support for Fibre Channel over Ethernet (FCoE). Even if the FCoE Converged Network Adapter (CNA) presents itself as a normal HBA to the host, the use of this configuration with Microsoft Failover Clustering is not supported. With one exception - Two node cluster configuration with Cisco CNA cards (VIC 1240/1280) and driver version 1.5.0.8 is supported on Windows 2008 R2 SP1 64-bit Guest OS in vSphere 5.1 Update 1.
  7. The use of Round Robin Multipathing for your Path Selection Policy (PSP) is not supported.
  8. If you are deploying a hybrid physical node – virtual node Microsoft Failover Cluster the Physical Node can’t use Multipathing software.
  9. No support for VM snapshots, which is one of the reasons that vADP backups don’t work.
  10. No support for Storage vMotion due to the use of pRDM’s.

Some of the above restrictions, especially lack of vMotion and DRS support make it very difficult for cloud service providers that are using vSphere to offer Microsoft Failover Clusters as a service. The reason is obvious. One of the main benefits of having an Infrastructure as a Service is completely non-disruptive hardware upgrades and maintenance. This is not possible with Microsoft Failover Clusters with the current constraints. If cloud service providers wanted to offer a Failover Clustering option they would need to notify customers to shut down their cluster nodes each time the firmware, drivers, or hypervisor version needs to be updated on their hosts. This of course also applies in your private cloud. Downtime would be required on the nodes each time the hosts need to be updated due to the lack of vMotion and DRS capabilities.

Even with the limitation though the advantages of virtualizing your clusters still outweigh the drawbacks. You still benefit from VMware HA, and the performance and reliability you’ve come to expect. You also get the benefit of being able to use VMware SRM for Disaster Recovery.

Options and Alternatives for High Availability

Failover Clustering is inherently complex. It doesn’t always provide high availability either. There are scenarios where downtime is still required and that downtime might be as much as would be expected just using VMware HA. Because the underlying disks are shared any storage loss or corruption will affect the entire cluster. A cluster if very static and hard to move about between hosts or from your private cloud to a cloud provider if you wished. These are some of the reasons it’s not always the best option.

When considering clustering and you think you’re protecting against Guest OS or Host failure think about the last time you saw a Blue Screen of Death (BSOD) from a VM, or you had a host fail. Hardware reliability is greatly improved and most BSOD’s are caused by drivers. With the standard drivers used when you virtualize your servers you are very unlikely to get a BSOD, at least based on the VMware drivers. There will always be exceptions but this is the case based on my experience for the vast majority of workloads, including those with high availability requirements. Microsoft Failover Clustering is not a DR mechanism (generally), so you need additional measures to provide DR, however some of the alternatives can provide HA and DR in the one solution.

Microsoft Failover Clustering can provide flexibility at times around OS patching. But even this use case has alternatives that provide the same level of availability. I give you an option around rolling patch upgrades below.

If you want to provide high availability to vCenter and the vCenter components then the options are VMware HA and vCenter Server Heartbeat. If you are looking to provide high availability to SQL Databases (ones that are not being used for VMware products that don’t support database clustering) then you have a number of options and alternatives, again these are in no particular order.

  1. In guest iSCSI initiation. This is fully supported and will still allow vMotion and DRS migration to occur. Please Refer to the VMware Guide Titled - Setup for Failover Clustering and Microsoft Cluster Service – Update 1, ESXi 5.1, vCenter 5.1. The guide reads as follows on page 9 “Use of software iSCSI initiators within guest operating systems configured with MSCS, in any configuration supported by Microsoft, is transparent to ESXi hosts and there is no need for explicit support statements from VMware.”   Although VMware hasn’t specifically tested this with Windows Server 2012 Failover Clustering there aren’t the same restrictions as this is relying on standard in guest support for the clustering, which as per the guide is transparent to ESXi and does not require any specific support statements from VMware. Provided Microsoft Supports it (Direct Guest Initiated iSCSI for Windows Failover Clustering), which they do, then it’s fine. I would still recommend in guest agents for backups in this case. Incidentally Cisco has a great guide on how to deploy this configuration - Microsoft SQL Server 2012 Failover Cluster on Cisco UCS with iSCSI-Based Storage Access Deployment Guide. This option allows more than the 5 cluster nodes supported by vSphere normally and in fact you could configure a cluster up to the maximum number of nodes supported by Microsoft.  This option is not supported with the use of VM snapshots. You can read more about Snapshot Limitations Here.
  2. If you’re wanting high availability above 99.9% for an application such as Exchange or SQL Server you can use the built-in replication technologies such as DAG’s, Database Mirroring, Log Shipping, or Always On Availability Groups depending on the version. These are fully supported by VMware, have full support for vMotion, DRS and HA, and can also provide a DR mechanism. They also are supported with use of VMware vSphere API’s for Data Protection (vADP) for backups. DAG’s and Mirroring or Always On Availability Groups can be used for high availability as well as disaster recovery. The failover can also be completely automated. They also provide additional protection against disk based corruption where clustering would completely fail. You should check if your software vendor supports the Microsoft SQL Client (if using SQL Server) and these automated failover options. Unfortunately VMware doesn’t support Database Mirroring or Always On Availability Groups at this time for SSO or vCenter databases.
  3. You could choose to keep things simple and just rely on VMware HA. This is a very viable solution for up to 99.9% availability. This is a great solution for the vast majority of cases. I know of a single VM with 528GB RAM and 32 vCPU’s being protected by VMware HA and it runs the entire SAP system and Oracle DB for a very large organization and has done so reliably and performed exceptionally well meeting their SLA’s. When at all possible I recommend keeping things simple. Unjustified and unnecessary complexity adds the risk of downtime and higher probability of human error.
  4. If you need more availability than VMware HA alone can provide  you could add VM and Application Monitoring and Application HA. This will cover the cases of individual application services failing within the guest.
  5. To cover the use case of failover during in guest patching you can use vCenter Orchestrator in combination with hot add and hot remove and clone operations of virtual disks to patch the OS disks or application disks of a single VM while it’s still running and then fail over to the patched version. This requires some advanced understandings about how the OS and hypervisor work together and would best be done along side VMware PSO, but it is possible. This would achieve very similar availability profile during the rolling patch process as Failover Clustering would.
  6. If you wanted to build a Microsoft Failover Cluster inside of a VMware vCloud Director vApp you could also achieve this. You would need to create the cluster nodes connect them to the shared storage by using in guest iSCSI initiators. They could either connect through an external network out to the iSCSI storage or you could make an iSCSI Target VM as part of the vApp. This would give you Microsoft Failover Clusters as a Service inside an Infrastructure as a Service environment running on top of vCloud Director, with self service, and on demand. There would definitely be some scripting involved but this could be a viable solution, and with orchestration you could also set appropriate anti-affinity rules each time one of these clusters was deployed. No restrictions on HA, vMotion or DRS, it would just work. This option allows more than the 5 cluster nodes supported by vSphere normally and in fact you could configure a cluster up to the maximum number of nodes supported by Microsoft. This option is fully supported by VMware. The idea of using an iSCSI VM inside a vApp came from Andrew Mitchell (@amitchell01 also a VCDX) a colleague from the VMware APJ CoE. This option would not support VM Snapshots. Supportability of vADP for backups is unclear as vADP does get around some of the same limitations for snapshots. But this is not recommended as an option for Production vApps, only development and testing. 

 

Final Word

This article covered the current status as of vSphere 5.1 GA. It is very likely that improvements will be made in future releases to address some of the limitations highlighted above. VMware understands what it needs to do in order to deliver the Software Defined Datacenter and to support Business Critical Applications. I’m sure they’re already working hard to improve platform support for Microsoft Failover Clusters, even if it is only needed in a very small minority of cases. In the meantime my recommendations are to use VMware HA and VM Monitoring or App HA unless there is a very strong justification for something in addition to this. If you have that strong justification then leverage the built in application high availability and protection options. Failover Clustering due to it’s complexities and risks is a last resort and inferior in most cases to application level HA.

[Updated 10/09/2013] As of vSphere 5.5 Microsoft Windows 2012 Failover Clustering is fully supported using Fibre Channel, FCoE, iSCSI or any of the in-guest storage IO access methods. Failover Clustering is also supported for the vCenter Database as of vCenter 5.5.  I cover the enhancements to clustering in more detail in a separate article – vSphere 5.5 Windows Failover Clustering Support.

I’d be very interested to get your feedback on this article and hear some of your experiences running Failover Clusters in VMware vSphere.

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.comby Michael Webster +. Copyright © 2013 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

  1. March 23, 2013 at 5:00 pm | #1

    Hi Michael,

    For some of my costumers i'm actually recommending using a dedicated ESXi HA cluster. Of course you need the appropriate justification for this kind of design decision as it depends on many things such as SQL capacity needs, budget, etc.

    I do believe separating the SQL cluster to a dedicated ESXi cluster is a good solution if possible.

    Because of the fact the Physical Mode RDM's comes with so many limitations I love to isolate them, it gives much greater maintenance windows on the non-SQL-Cluster environments and a peach of mind.

    Personally, I hate RDM's of any kind so the fact the SQL 2012 can work with DAG is a blessing.

    In terms of protecting the vCenter DB I found the Heartbeat as the best solution but it's a bit expensive though.

    • @vcdxnz001
      March 23, 2013 at 5:06 pm | #2

      Hi Lior, I agree in a lot of cases a dedicated VMware HA Cluster for SQL workloads is a good idea. Whether they are using Microsoft Failover Clustering or not. One reason this is popular and will become more popular is due to licensing. You can just license the whole cluster for SQL and run as many DB's as you like within physical hardware limits. One thing to watch out for though is too much overcommitment of resources if you have jobs such as log shipping or other scheduled tasks that all happen at exactly the same time. But provided you've handled this aspect it's a great design choice in a lot of cases. Thanks very much for your feedback.

  2. Bozo Popovic
    March 24, 2013 at 9:32 am | #3

    Hi Michael,

    i am not sure does this apply to 5.1 version but previous version had limitation with lack of support for performing VMware snapshots with in guest iSCSI connectivity. It makes a lot of sense why that was not supported therefore i believe that limit still persists. If that holds to be true, ho do we have a support for VADP backup solutions?

    Thanks,

    Bozo

    • @vcdxnz001
      March 24, 2013 at 9:17 pm | #4

      Hi Bozo, if you cluster is connecting out to an iSCSI array then you won't be able to snapshot or use vADP, but if your cluster is connecting to an iSCSI Target VM then you will be able to use vADP and Snapshot. Provided of course you aren't using Fault Tolerance to protect the iSCSI VM. You can't snapshot an FT VM.

      • Bozo Popovic
        March 25, 2013 at 9:42 am | #5

        Hi Michael,

        sorry, maybe i was not clear in my statement. I meant to say that cluster nodes (vm's) of any shared disk clustering solution with in guest sw iscsi are not supported to be backed up with VADP backups only with in-guest agents solution.

        This sometime can create a problem if a customer has already strategically aligned himself with VADP based backup solution.

        Thanks,

        Bozo

      • @vcdxnz001
        March 25, 2013 at 10:17 am | #6

        Hi Bozo, If your failover cluster nodes and your iSCSI storage are all VM's then you could snapshot them and you could back them up. Especially if you consider using Windows Server 2012 as your iSCSI target as you could then even use VSS to quiesce the filesystems. I haven't tried this but it would work in theory. This being the case why would vADP then not work? The cluster wouldn't even be aware of what is happening. I wouldn't recommend this for high throughput production systems, but it might work for Dev / Test. If a customer has strategically aligned with vADP they have a choice to use Microsoft Failover Clustering or a choice to not use Microsoft Failover Clustering and make use of vADP. At the moment for production workloads the choices are mutually exclusive.

      • Bozo Popovic
        March 27, 2013 at 4:33 am | #7

        Hi Michael,

        thanks for reply. I believe we have to work with and around those limitations/restrictions for now and find our way to meet the enterprise goals for backup.

        Thanks once again,

        Bozo

    • Bozo Popovic
      March 25, 2013 at 11:18 am | #8

      Hi Michael,

      a lot of our solutions go through validation and upon such validation satisfies the criteria we issue an official support statement. I believe we do not support in guest iscsi with snapshots mechanism and thus preventing support for VADP backup solutions. Unless the VADP uses completely different mechanism to execute snapshots other than VMware snapsot

      http://pubs.vmware.com/vsphere-50/index.jsp?topic

      …and a bunch of other kb articles.

      http://kb.vmware.com/kb/1025279 http://kb.vmware.com/kb/1009402

      I agree that it may work but this should not be recommended in production, right?

      Thanks,

      Bozo

      • @vcdxnz001
        March 26, 2013 at 9:41 pm | #9

        Hi Bozo,

        Great points and thanks for links to those articles. I agree 100% that the use of snapshots of vApps with in-guest iSCSI, even if the iSCSI storage VM is inside the vApp is not recommended for production use. I also agree that there is a reason for VMware's validation processes. If it's not documented it's not supported. The theory behind the reason for not supporting snapshot of VM's with in-guest iSCSI is that we have no visibility or control over the iSCSI array that is hosting the VM's disks. However this would not be the case if the iSCSI Target is part of the vApp and a VM itself. Also I would like to draw your attention to the following line that is described in the VMware documentation that you've linked to:

        "Backup solutions, such as VMware Data Recovery, use the snapshot mechanism to freeze the state of the virtual machine. The Data Recovery backup method has additional capabilities that mitigate the limitations of snapshots."

        In this case, based on this documentation, you would be encouraged to use vADP as a possible backup mechanism. At best this is a little bit of a gray area.

  3. Chris Williams
    March 24, 2013 at 2:44 pm | #10

    Michael – Good article as usual. …But I have to tell you that, unless he told you this back in the vSphere 3.5 days, I was deploying the iSCSI Target VM configuration/method before your friend Andrew suggested it to you. :-)

    I've had great success with database clustering using this configuration for a long time on vSphere starting with vSphere 4.0 about 4 years ago. Back then, we deployed Oracle RAC in production for a large client using it on a fully converged, blade based network using Lossless 10GbE and no FC SAN (so we essentially had iSCSI on iSCSI).

    I first spoke of this publicly at the Oracle on vSphere "Dream Team" Panel back at VMworld 2011 (class BCA1548), referring to it as a iSCSI Gateway VM. We did things that way back then because, even though guest to host storage with iSCI eliminated SCSI bus sharing, going guest to guest gave us far superior flexibility and consistency with our VMs.

    Lots of folks don't realize the capabilities of guest-to-guest iSCSI solutions. You can use pretty much any iSCSI target you want as long as it supports the iSCSI needs of your cluster (ex: MSCS needs SCSI3 Persistent Reservations). You also need to do quite a bit of iSCSI tuning for performance. You should share the iSCSI gateway network on a private Portgroup for your database cluste for security reasons. The gateway VM itself should be configured to use VMware Fault Tolerance (yes a single vCPU is enough to deliver the performance you need). Also, as with any iSCSI based storage solution, don't forget to turn on Jumbo Frame support.

    But the end result is a database cluster with no SCSI bus sharing and all storage neatly tucked into VMDK files on the vSphere Datastores of your choice. This has the added benefit that your data protection mechanisms for your database clusters can use the same tools as all of your other VMs. It also runs on every supported vSphere hardware configuration and is easily portable. Finally, it does not have any of the ESX host limitations seen with other solutions – you can use this on a 32-node DRS cluster if you like.

    Best of all, this is absolutely a VMware supported configuration. I have statements from VMware stating such (happy to share them with you if you like).

    And as I mentioned before, I've successfully deployed this configuration for both Oracle RAC and SQL Server clusters for several clients. It's a terrific alternative. Ironically, it also looks impressively like what VMware is doing with Virtual Volumes.

    By the way, with Oracle RAC, you will run into another issue in a vCD environment. Even with the current 11.2.0.3 version, basic scripting doesn't help you with everything. The issue isn't just scripting to change IP addresses for the target. Clusterware itself freaks out if you just arbitrarily change the IP address of your cluster nodes – be they physical or virtual. As it turns out, changing the IP address of an Oracle RAC node is basically unsupported. There are some ways around it, but it's not as straight forward as it looks. For example, you can still provision a cluster of nodes that are ready for a scripted RAC install.

    Anyway, nicely written article.

    Cheers,

    Chris Williams

    • @vcdxnz001
      March 24, 2013 at 9:33 pm | #11

      Hi Chris,

      I have scripts that allow the automated provisioning of a RAC database that can be used with vCD already. I would need to update them to include the iSCSI Target API's to automatically provision the iSCSI storage to the RAC Node VM's. You're right about it not being as simple as changing IP address, and the scripts take account of that. But in any case in a vCloud environment, which is what I was referring to with option 6 there is no need to change ip addresses when a new vApp is deployed. You can use the same IP addresses and configuration as it had before and stand up multiple copies if you like. All because they are fenced off using vCloud Networking and Security. If I was creating a training lab I think this is how I would do it.

      In any case the option is there also for Microsoft Failover Clusters to do the same thing inside of vCloud Director. However you wouldn't be able to use FT as that's not supported by vCD. If you were deploying just to vSphere you wouldn't be able to take snapshots or vADP backups if you were using FT because snapshots and vADP backups are not supported with FT. But the idea of using FT for the iSCSI target VM is a good one and this configuration certainly gets around limitations of deploying Microsoft Failover Clusters directly to vSphere. Deploying clusters to vCloud Director would not be possible without external iSCSI array or an iSCSI Target VM. Although with RAC you could do it easily enough using dNFS also either to an external NFS target or an NFS VM inside the vApp.

      Definitely good to leverage Jumbo Frames, especially if on a 10G network. It's great that these don't suffer from any of the limitations of deploying directly with vSphere and will support the maximum config that Microsoft Supports. No bus sharing and the ability to use VMDK's are all great. But there is an increased overhead in management that needs to be understood and accepted also. There are tradeoffs with every solution.

      Thanks for your feedback and contributions they are greatly appreciated.

    • DMF
      March 27, 2013 at 9:13 am | #12

      Hi Chris,

      Is an example of an iSCSI Gateway VM a linux VM with iscsid installed to present the storage to the DB engine nodes? And then have DB engine nodes connect with inguest iscsi initiators? I use this setup in production for RAC archivelog targets.

    • DMF
      March 27, 2013 at 10:22 pm | #13

      Hi Chris,

      Is an example of an iSCSI Gateway VM a linux VM with iscsid installed to present the storage to the DB engine nodes? And then have DB engine nodes connect with inguest iscsi initiators? I use this setup in production for RAC archivelog targets.

      (Sorry, I replied to Michael by mistake)

  4. Chris Williams
    March 25, 2013 at 1:06 am | #14

    Cool – So you are then scripting a shutdown of everything but Clusterware and then doing a crs reconfigure plus a few other things that need to be modified. It's possible to do, but it's just something where I would include a "don't try this at home, kids" kind of disclaimer. I agree that fencing in a non-production or training environment on vCD is another nice way around this issue.

    I tend to be somewhat less of a vCD advocate for production environments though. I'm not saying you can't or shouldn't use it in production, but the main things that are huge advantages for vCD (easy, controllable, rapid change) are just the things that make a lot of operations folks shudder in production. …especially that "rapid change" part. The limitations we're talking about here also make it a little less than 100% production environment friendly in my opinion. But since more than 75% of changes happen in non-production environments, vCD addresses an incredibly important segment. I do expect it to continue improving over time.

    Actually, the way we usually got around that you cant backup a FT VM via vADP is to script turning off FT prior to doing the backup, then automatically turning it back on again after the backup is complete. There are lots of scripts around for doing this. Yes, you are then in an unprotected state FT-wise during the backup, but that's usually an acceptable level of risk.

    Hopefully vSphere.Next will finally clear up this issue and (we can always hope…!) launch SMP Fault Tolerance as well. Once that's out, it's a game changer for clustering. We basically won't need MSCS or RAC at all except perhaps in some very "corner" cases.

    Chris

    • @vcdxnz001
      March 25, 2013 at 3:09 am | #15

      Hi Chris,

      Correct, the scripts and orchestration via VCO was demonstrated by Bryan Wood during VMworld 2012 in APP-BCA1333. It's not something you'd want to attempt to do without VMware PSO involvement or a lot of testing. The scripts include the functionality to add RAC nodes on demand to a running RAC cluster also, as well as adding disks where needed, so it's quite flexible. Just the iSCSI API integration that hasn't been done. There is a downside though and that is you need a call out from vCD to automate the creation of anti-affinity rules for the RAC cluster nodes via AQMP and vCO. This would be implemented as a blocking task.

      Also I'm starting to see a lot more use of vCD in production use cases for certain scenarios as the admins no longer have to worry about setting up reservations to meet the App SLA's on a per VM basis as vCD handles that automatically. But these are still treated as high governance environments where proper change controls etc are in place, so therefore are less rapid, more admin self service than end user. Have done a bit of load balancing integration and automated scaling to handle peak loads of different app / web tier systems. But Database systems in vCD generally still use traditional backup, are fairly tightly change controlled and then benefit mostly from the resource guarantees that are inherited from the allocation models of the Org vDC that they are deployed into. The ability to burst capacity into other cloud environments is also starting to be used by some customers. But this is early days. This has started to happen more now that a couple of the backup vendors support vCD with their solutions, including the use of vADP.

      vCD can also be used no problem if you're doing in guest agent based backups and where VM's don't require complicated vApp network configurations, which most production systems won't. But the main use cases are still Test Dev and the lifecycle management combined with vApp isolation is where the main ROI is right now. You can run multiple project streams in parallel without impacting each other and speed up time to market as a result.

  5. Garret Black
    April 3, 2013 at 3:15 pm | #16

    You bring up a good point with simply relying on VMware HA, but it would be really nice to see VMware step it up to allow multiple vCPU's with VMware FT. I assume they just weren't ready for GA with the extremely variable traffic that the FT logging would create. Apparently it was going to be in 5.1 but I have yet to see any updates for FT in awhile (even under NDA).

  6. MN
    April 5, 2013 at 5:38 pm | #17

    Great article! My company is considering breaking up several existing 2-node virtual/virtual SQL Server clusters and making these standalone servers which rely on VMware HA. Have any of you folks done this? Do you have any regrets? The only potential downside I see is a slightly longer downtime for patch install reboots vs rolling patches in our current setup.

  7. sp
    April 25, 2013 at 3:14 pm | #18

    I think you will find that in some cases that Windows Failover Clustering (WFC) will provide best HA solution for SQL Server; especially in those environments that cannot stomach outages due monthly OS patching (plus it not uncommon for a SQL service pack install to take 30-45 minutes to install). For this reason, I am thinking that VMWARE will eventually support WFC 2012.

    1st. log shopping is not a true HA solution for SQL (MS has even stopped calling it HA) because it does not provide automatic failover and when the database is manually failed-over the app does not transparently connect to the log shipped copy (the app sql connections strings need to be updated). Log shipping can best described as database redundancy solution (i.e. full database data replication solution).

    2nd. Database mirroring will be deprecated in the future (http://msdn.microsoft.com/en-us/library/ms143729.aspx ) so I don’t think there is value to discuss things that will be going away.

    3rd. AlwaysOn Availability Groups feature, a.k.a. HADR (some compare it to MS Exchange DAG) requires Windows Failover Clustering. Also, Availability Groups requires the user databases (this technology does not support system databases) to be in the FULL RECOVERY model – some database apps (e.g. data warehouse) do not want the performance overhead related to FULL RECOVERY model setting so in these WFC may be the best SQL HA solution.

    • @vcdxnz001
      April 25, 2013 at 9:59 pm | #19

      Hi Sp, Not all options for SQL Server availability in SQL 2012 require bus sharing or RDM's. You can have automatic failover between members of the AlwaysOn Availability Groups without traditional Failover Cluster Instances. So this is quite different to the AlwaysOn Failover Clustering Instance, which is more like your normal failover clustering with shared RDM's. This is why I have mentioned it here. Also you don't need to have WSFC2012 to support AlwaysOn Availability Groups, so it can be used now with WFC2008 and SQL Server 2012. This reduces the impact of having no support for WSFC2012 on vSphere 5.1. I agree eventually VMware will support this as there is a big focus on Business Critical Apps. I wouldn't recommend you make architectural decisions now on what might happen, but rather on what can be implemented and what is supported. While mirroring might be going away it's still a valid option now.

      I have recently been talking to Symantec about their new version of VCS Clustering, which is completely vSphere aware, works with VMDK's, doesn't have any of the scalability issues of WSFC on vSphere. This looks to be a viable alternative to WSFC until better support is provided. It would also allow service providers to provide a real Failover Clustering as a Service for customers while still being able to protect and maintain the platform without disruption to customer workloads. Full vMotion, DRS and HA support out of the box. I will write a separate article about this solution when I get time.

      • SP
        April 26, 2013 at 1:40 am | #20

        True SQL2012 Availability Groups work with WFC2008 but I was focusing on WSFC2012 VMWARE support. True SQL2012 Availability Groups does not require a SQL Server Failover Cluster Instances (FCIs) & hence no need shared storage (i.e. RDMs) but SQL2012 instances enabled for Availability Groups are required to be hosted on servers that are node members of a Windows Failover Cluster (e.g. whether it be win2k8 or win2k2012) – the take home here is you are not going to avoid WSFC by using SQL Availability Groups.

        I guess when I look at “high” in HA, I expect the HA technology to have some important key attributes such as continuously operational, ensure uninterrupted data flow and operability, minimize the impacts of the failure and maintain availability to a degree that the apps/users are not affected, etc. – automatic failover during unplanned outage is important in that regard.

        Still Database mirroring/Availability Groups – require FULL RECOVERY model – some database apps are unwilling to take the associated performance hit – WSFC fills this gap nicely were database are in the bulk-logged or simple recover model. For Database mirroring/Availability Groups automatic failover then Synchronous mode will need to be enabled – again another potential performance hit to support this 2-phase commit like process – both your storage on all servers and network must be well tune for a high transaction database, i.e. SQL transaction log best practice write I/O latency is data mirror transfer over network –> mirror partner t. log write) must fast else app users will feel the slow-down on database writes.

        With database mirroring/Availability Groups sometimes its not cost effective to have a redundant copy for very large database, e.g. we have a single database that is 65 TB. O&M with database mirroring (& Availability Groups to a much lesser extent) can be laborious compared to WSFC, e.g. take environment were Window Admins have to patch 100 of server hosting SQL per month (i.e. monthly patching cycles), say one of those servers has a single SQL instances with 50 user databases that are database mirrored, I can tell you from experience that a Windows admins in not going to want to have to failover each user database via SQL tools or SQL scripts (or have to rely on a SQL DBAs) rather they would preferring WSFC & failover the entire SQL instance via Failover manager. Plus the SQL DBA O&M related to setup, maintaining, & monitoring of each database mirror can also be laborious. In small environments with few servers & SQL databases this may not be a big deal.

        Having a database mirrored redundant database copy is kinda ineffective in terms of HA unless its associated app can access it transparent manner after a manual or automatic failover to the mirror partner – having to update an app’s database connection string upon failover is not transparent. I think its fair to say that to have an end-to-end HA system one would expect that an app could transparently access its database layer after a database failover. With database mirroring that happens at the database driver level (e.g. SQL Native client via the ODBC, OLE DB, ADO.NET, etc. database providers/APIs) with database mirror failover partner attribute settings in the database connection string. Some database drivers simply do not support this database mirror connection routing and many COTS do not support modifying connection string setting for database mirror (e.g. Sharepoint 2007, SCOM 2007, etc.) and even state explicitly say in their docs that they do not support database mirroring. I am not trying to slam database mirroring because it definitely has it place but I am just trying to point out the gaps were WSFC is the best option.

        I think many want win2k12 for the new benefits & features (e.g. performance) and are surprised that VMWARE does not "officially" support WSFC2012. Its funny that I only found out that VMWARE did not officially support WSFC2012 until after we successfully setup a few non-prod single & multi-subnet WSFC2012 without any problems – obviously will not go production until there is official VMWARE support.

        I can't say much aboutSymantec ApplicationHA other than 3 years back we looked at a related product called VERITAS Cluster Server (VCS) for our physical servers and it was really expensive and we did want to get have to get locked into their Storage foundation, e.g. did not want to give up the Windows disk manager for their volume manager.

      • @vcdxnz001
        April 26, 2013 at 4:07 am | #21

        WSFC is a good solution to the right problem. But far too often it's implemented where it's not even needed. Standard VMware HA and/or AppHA if required (whether you use third party plugin or write your own healthcheck with vSphere 5.1), can handle most requirements where clustering would have ordinarily been used. Also huge monolithic database servers are also overused where multiple instances or schemas are consolidated down onto a single system. Often this is not efficient from a change management, availability or recoverability perspective. But with every workload the right availability design will depend on the requirements. But none of these options is fully non-disruptive. Even with WSFC you have a small outage during the failover between nodes. If you want true non-disruptive high availability and always on applications you need to go to Oracle RAC or similar and build this into the application layer, which as you quite rightly point out not all applications, especially COTS applications have done. WSFC's achilles heel is that it can't protect against data or disk corruption where other forms of availability protection can. It's always a balancing act choosing the right solution. Clustering is only needed for a very small percentage of database servers and applications and should only be implemented for the systems that need it.

        Might be worth another look at AppHA from any of the vendors and VCS, a lot has changed in three years. Three years ago vSphere could only do 8 vCPU per VM, now it can do 64. A lot's changed with Veritas Cluster Server also. But you don't need to buy third party software to get AppHA, you can build your own healthchecks and services now with vSphere 5.1.

      • SP
        April 27, 2013 at 7:01 pm | #22

        “But far too often it’s implemented where it’s not even needed. Standard VMware HA”, correct – no sense to WSFC sql unless the app-to-db end-to-end is HA; stand-alone SQL VMs are much easier to O&M. “But none of these options is fully non-disruptive. Even with WSFC you have a small outage during the failover between nodes”, correct.

        “If you want true non-disruptive high availability and always on applications you need to go to Oracle”, I would say Oracle’s “"rolling upgrade" is a zero-downtime” claims are a farce – especially when having to the patch OS RAC nodes. Our org is big time into Oracle RAC and we are in the top 3 owners of Oracle Exadata appliances in the world–it’s not an issue with our staffs skill-set given that Oracle support is onsite. Definitely RAC zero-downtime has not been realized in our environment to our management’s dismay ; our SQL meets or exceeds the RAC. Being both an Oracle & SQL person, I get both have their place in the enterprise. Take a look at Microsoft’s “Why Oracle RAC Might Not Be Suitable For You” – of course you have to consider the source but I think it’s fair.

        “WSFC’s achilles heel is that it can’t protect against data or disk corruption where other forms of availability protection can.” true but the same can be said about Oracle RAC (that’s why the RAC + Data Guard tandem is commonly used for the most critical systems); in our environment our mostly critical system use WSFC FCI with a plus-up of database redundancy via log ship, database mirroring, or availability group. To be honest I have not experienced database corruption in years ever since we moved away from direct-attached storage (i.e. mostly cause by RAID controller issues) to SAN.

        For those whom may not know about Availability Groups (AG), if the cluster SQL Server Availability Group resources goes offline or the cluster goes offline (i.e. clussvc.exe ) then your Availability Databases will become unavailable. Personal I don’t get why MS wants to depreciate database mirroring (or having something similar w/o a WSFC dependency) – given that they spent so many years promoting it as GO-TO and a lot of the SQL DBA community followed & invested heavily in it. Ideally database mirroring should have be enhanced to include the new features from AG, i.e. multiply replicas, read-only replicas, backup replicas, & grouping many mirrored database for management as a single unit. Just saying.

        “Might be worth another look at AppHA”, true – I need another look.

        FYI, some may find this helpful. In the past we had stability issues w/ WSFC 2008 R2 on VMware vSphere 5.1 (but none with our physical WSFC) – the WSFC would randomly crash 1-2 per month – the errors looked to point to storage, i.e. quorum. Our virtualization, network, storage, & Windows teams gave the infrastructure a clean bill-of-heath; MS Premier support had no clue. Of course we follow the VMWARE best practice articles that you list above, plus the VMWARE ones for SQL.

        The details of ours incidents are near identical to the one others have experienced here:
        http://social.msdn.microsoft.com/Forums/en-US/sql

        The resolution solution was a combo of:

        Apply all hotfixes in KB2545685 (most important: KB2545685 & KB2552040)

        Disabling the IP Helper Service (IPv6 is already not bind to the NICs but this does not disable IPv6 )

        Apply Chimney settings *

        Apply cluster heartbeat settings*

        * Chimney settings and cluster heartbeat settings specified by Stew – see Stew posting on Monday, September 26, 2011 1:17 AM

      • @vcdxnz001
        April 28, 2013 at 1:27 am | #23

        Thanks SP, that is very helpful info, I hadn't come across those issues so it's great to know, I'm sure the other reads will greatly benefit from it as well. I agree with your take on RAC also. The complexity alone often introduces risk of increased downtime. Often you'd need to evict a node in order to do a non-disruptive upgrade, but problem being reintroducing a node or brining a service back online can itself cause some disruption. With a combination of Oracle RAC, Dataguard and Fast Start Failover I've managed to get a system running without downtime during upgrades and changes. But this is a highly complex configuration to operate and manage. It required a lot of testing and a lot of training before the DBA's could operate and manage the system. Also we had to make sure the applications and databases we were running could operate in this environment, so a considerable amount of application testing and tuning as also required. But the complexity and effort was justified by the particular customer requirements and constraints. Again thanks for the feedback, it is greatly appreciated.

        I'd be interested to hear your thoughts on what feedback you'd give to the VMware Product Managers that look after Clustering Support (Availability) and also other aspects that impact business critical applications. If you had a direct line to the Product Managers what would you tell them you need in vSphere? What should VMware improve as a priority?

  8. Arjuna
    April 25, 2013 at 4:21 pm | #24

    I think how Failover clusters differ from VMWare HA is that the Failover clusters provide application-aware HA whereas VMWare HA provides only hardware-level HA.

    We are too planning on going virtual for our existing SQL clusters running on physical hardware but still undecided as to which technology to go with.

    I have looked at Symantec ApplicationHA for VMWare which seems to bring the application-level HA back into the picture on top of VMWare HA. Has anyone had any luck with that solution?

  9. Andrew Firth
    May 23, 2013 at 11:51 pm | #25

    Fantastic Article Mike which answers all of my questions

  10. Clive
    May 30, 2013 at 2:16 pm | #26

    Mike, you have incorrect info about VMotion being supported with in-guest iSCSI setup of MS Clusters. "In guest iSCSI initiation. This is fully supported and will still allow vMotion and DRS migration to occur." Vmware KB http://kb.vmware.com/kb/1037959 is crystal clear that this is not supported.

    • @vcdxnz001
      May 30, 2013 at 3:13 pm | #27

      Hi Clive,

      It's not actually that clear cut. But thanks for brining up that KB as I've just reviewed it again and noticed it's been updated since the last updates I requested. It now includes FCoE support for Cisco UCS in certain configurations from vSphere 5.1 U1. This is good news. I'll seek further clarification on in-guest iSCSI and post an update here. The reason I say that is because in-guest iSCSI is not using vSCSI Bus Sharing in the virtual machine and that is what causes vMotion to be unsupported. Watch this space.

    • @vcdxnz001
      May 31, 2013 at 5:51 am | #28

      I got the KB updated to be more clear around the in-guest iSCSI support. So the in-guest iSCSI option now explicitly states that vMotion is not supported. You'll notice the KB is now updated if you go back and review it. I've updated this article to align with the KB. I'm still seeking clarity on why it's not supported and will update further if possible. I can't see any technical reason why it wouldn't work.

    • @vcdxnz001
      May 31, 2013 at 2:35 pm | #29

      Ok, this has gone around in circles a bit but looks like in-guest iSCSI is supported with vMotion as I originally had stated. Will update the article shortly again and am getting the KB modified again. Also seeking further clarification to ensure consistency in documentation.

Comments are closed.
%d bloggers like this: