| Unique Visitors |
Powered By 
| Unique Visitors |
If you have an application that needs very high service levels for availability (99.999%), 24/7/365, including maintenance and patching, then OS Clustering with shared storage is a proven solution. However it can be very complex to set up and maintain Fibre-channel, iSCSI or direct attached shared SCSI solutions. In some cases complex configurations are required not just in the hardware, but also the Operating System of the Guests. If you add in virtualization to the mix, the complexity level can increase more, with the need for physical mode raw device maps. Ironically, increased complexity can decrease overall availability, especially with an increase in the probability of human error. So how do we increase availability, decrease complexity, and provide a simple solution for OS Clustering with shared storage?
Nutanix AOS 5.17 with AHV has the answer. Nutanix AHV allows shared storage for Guest OS Clustering without any complex back end storage support or configuration, unlike other hypervisors from leading vendors, which still require Fibre-channel storage if you wish to use virtual disks. From Nutanix AOS 5.17 onwards you are able to configure a shared volume group and directly attach it to 2 or more VM’s and set up Guest Clustering without any complex in guest OS storage configuration at all. There is no complex storage back end, as that is all provided automatically by Nutanix AOS, and no in guest storage configuration, that might ordinarily be required if using iSCSI. This makes the use of Guest Clustering incredibly simple, as well as being very easy to automate, and significantly less difficult to support and troubleshoot.
The process for creating a Guest OS Cluster has 3 main steps:
Here is an example of how the Volume Group might look in the storage section of Prism for AHV:
While it’s possible to have up to 256 vDisks or Volumes within a Volume Group it is recommended to have 32 or less. If you need more Volumes you can create more Volume Groups.
When you attach a Volume Group to VM’s they will be listed in the Volume Group page within the Storage section of Nutanix Prism Element as follows:
If you wish to have a mixed virtual + physical cluster you can choose to enable external client access to the Volume Group. Any physical / external clients can then use iSCSI Initiator to connect to the clusters Target Data Services IP (DSIP) and mount the volumes.
In the example above I created a 4 node Windows 2016 Cluster, which will host SQL Server 2016 as the primary application. The VM’s are listed below, along with an AD Domain Controller:
After the Failover Cluster Manager components and tools are installed you can configure the Failover Cluster. Note: as part of the cluster creation a verification wizard is executed to ensure compliance with the strict rules needed to form a cluster, including shared storage tests for SCSI fencing and persistent reservations. The nodes in this case were displayed as follows within Failover Cluster Manager:
The next step is to install SQL Server on the cluster nodes and assign all the necessary dependent resources, which would look like the following:
I installed a second SQL Server instance in the same Failover Cluster so I could do comparisons between different configurations. You can see that in the image below:
After I created the cluster I did a series of tests including using tools such as HammerDB and Benchmark Factory for Databases. During the tests I performed live migrations to ensure the cluster didn’t blink in spite of the load, and it worked flawlessly.
Final Word
Nutanix AOS 5.17 and AHV makes creating guest clusters simple and quick and supports both Linux and Windows Guest OS types. You can now configure your fav clustering solutions without the traditional complexity and that means it’s way easier to automate. A Nutanix AHV cluster can now support any number of cluster nodes supported by the OS vendors. The next step in the evolution of this will be when AHV supports Metro Cluster across sites, along with volumes, which will allow for geo distributed guest clusters with greatly reduced complexity compared to the traditional implementations. The cluster example in this article with Windows 2016 and SQL Server 2016 was created just by following the standard Microsoft Documentation and directly attaching a Nutanix Volume Group on AHV directly to the 4 Windows VM’s that would form the cluster. That’s it, no special tuning or complexity needed.
This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster. Copyright © 2012 – 2020 – IT Solutions 2000 Ltd and Michael Webster. All rights reserved. Not to be reproduced for commercial purposes without written permission.
HammerDB is a very popular benchmark tool for testing multiple different database engines, including Oracle, SQL Server and PostgreSQL. This is a brief article to bring your attention to some ways you may improve your results and get more valid benchmark data.
We do a lot of testing with HammerDB at Nutanix. We run multiple database engines through HammerDB for each release of software we release. One of the great things about running databases on Nutanix is that once you find out how much performance you get from a given configuration of a node or database, you can scale it linearly and get the same experience. This fact was demonstrated by a test that Gary Little (Performance Engineering at Nutanix) did some time ago and published in his article SuperScalin’: How I learned to stop worrying and love SQL Server on Nutanix – Recommended Reading. Over the years we’ve found some things when using HammerDB that can improve consistency and reliability of test results.
Sometimes SQL Server might only use a single NUMA node due to the way connections and transactions are assigned in the engine. Gary describes the problem and solution in his article SQL Server uses only one NUMA Node with HammerDB. The solution is a slight modification to how the HammerDB scripts execute.
Another issue we’ve frequently run into when testing large SQL Server databases that needs lots of cores and memory is that not all the cores get used. Often you might find only 20 CPU’s are being used by the SQL Engine, even though you are using Enterprise edition. This is due to the ISO and installer being used not being the Core Edition of SQL Server Enterprise. You need Core Edition to allow more than 20 CPU’s to be used by the database engine.
Another common issue that we have discovered and Gary has documented in his article HammerDB: Avoiding Bottlenecks In Client, is settings in the actual HammerDB client. Specifically around how the client logs data. Check out Gary’s article for the settings you should uncheck before running a test.
Other basic issues we’ve found is the system under test not being sized properly for SQL Server, or the VM’s running SQL Server not being properly aligned to the NUMA nodes and configurations of the hardware being tested. an 8 vCPU VM isn’t optimal if running on a 12 CPU socket, whereas a 6 vCPU VM would be better. We have found that right sizing the VM’s produces far better results. In this case, 8 vCPU’s might produce less transactions per minute and less new orders per minute and higher response times than if 6 vCPU’s was used. On an 18 core socket, 9 vCPU’s or 6 vCPU’s are also good options. Some people find it difficult to use an odd number of vCPU’s, but the scheduler works fine.
Final Word
Benchmarking SQL Server with HammerDB can help determine what performance you might reasonably expect from a platform running different types of databases and show how a platform scales, but it isn’t exactly the same as your real world workloads. The best way of determining how your workloads will perform is by testing them on a the platform with test cases you have valid comparisons for and under similar conditions. Always account for maintenance tasks, such as backups, stats update, reindex etc and allow headroom for growth. I hope the above help you and wish you all the best benchmarking with HammerDB.
This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster. Copyright © 2012 – 2020 – IT Solutions 2000 Ltd and Michael Webster. All rights reserved. Not to be reproduced for commercial purposes without written permission.
I’ve been doing some work recently with SQL Server and Nutanix Acropolis Hypervisor on a older Nutanix NX3400 platform in my lab, as my other clusters are busy on other things. The testing involves cloning a number of SQL Server databases, running performance and functionality tests, and then destroying the VM’s and starting over again. The idea is to make one change each iteration and then be able to do comparisons between iterations (keeping a consistent control / baseline). Always being able to return to a consistent state for the Databases is important to have valid test data. To do this repeatedly and quickly using multiple VM’s would not be practical without automation. Similar sorts of scenarios could be used where multiple test environments need to be created rapidly and returned to a consistent state after each test run, such as with a large application project. With all of this being able to do as many tests as possible in a limited period of time is important. The more tests, the more results, the more conclusions, and in a project environment the more defects found earlier that get fixed and prevented from going into production and ending up as support tickets. So in this article I will provide the scripts I used to automate this process, and demonstrate just how rapid it is to clone 12 x 300GB+ databases for testing.
This NX3400 platform I’m using has 4 nodes, each node has 2 x SandyBridge 2.2GHz 8 Core Processors, 256GB RAM, a single 300GB SSD and 5 x 1TB HDD’s. It’s a couple of generations old now, and has long since been replaced by newer and more powerful platforms. But it’s good enough for the tests I’m doing. Here is a screenshot of the cluster:
As you can see the cluster is running Nutanix Acropolis Hypervisor (KVM), and at the time of the test High Availability was not enabled, hence it is in yellow. The cluster is running Nutanix OS 4.1.4.
The Nutanix Acropolis Hypervisor is based on KVM, but has been customized to meet the unique requirements of a distributed linearly scaleable hyper-converged infrastructure platform, and to be as simple as possible to use. One of the major benefits for everyone from Nutanix Acropolis Hypervisor is that you don’t need to be a rocket scientists or computer science PhD to use it, whereas with standard KVM, that’s how it feels. Standard KVM has way to many options, knobs and ways to get things wrong. In my opinion that is why KVM has never been as successful as other hypervisors. Nutanix Acropolis Hypervisor changes all that, itis so easy my 8yr old Son can use it and create his own VM’s. Nutanix Acropolis Hypervisor is available in both the commercial Nutanix platforms from Starter Edition Licenses, as well as the free Community Supported Nutanix Community Edition, which can run on just about any hardware, even nested virtualization. While it doesn’t yet support all of the features customers may have in other hypervisors, it is good for many use cases, and especially for applications where they have built in availability and are natively written to scale out and be highly available (including the likes of SQL Server, Exchange, Oracle and Oracle RAC).
Here is the script I used to rapidly provision 12 SQL Server Databases, which is executed from one of the Nutanix Controller VM’s:
sql_clone.sh:
#!/bin/bash
template="SQLW2K8R2-Template"
# To use batch provisioning uncomment startnum and endnum arrays and modify the values
#startnum=("11" "101" "201" "301")
#endnum=("100" "200" "300" "410")
startnum=1
endnum=12
vm_name="SQLW2K8R2AHV"
batches=${#startnum[@]}
startt=`date +%Y%m%d%H%M%S`
if [ -f vmclone.log ] ; then
rm vmclone.log
fi
for i in `seq 0 $((batches - 1))`
do
echo "Starting batch $((i + 1)) from ${startnum[$i]} to ${endnum[$i]}..."
DATETIME[0]=`date +%Y%m%d%H%M%S`
echo "Creating $vm_name from clone..."
echo "Cloning $template batch $((i + 1)) of $batches started at ${DATETIME[0]}..."
echo "Cloning $template batch $((i + 1)) of $batches started at ${DATETIME[0]}..." >> vmclone.log
completedtasks=`acli vm.clone $(seq -w ${startnum[$i]} ${endnum[$i]} | sed "s/^/$vm_name/" | xargs echo | tr " " ,) clone_from_vm=$template | grep complete | wc -l`
echo "Number of provisioning tasks completed = $completedtasks" >> vmclone.log
echo "Number of provisioning tasks completed = $completedtasks"
DATETIME[1]=`date +%Y%m%d%H%M%S`
echo "Cloning $template batch $((i + 1)) of $batches finished at ${DATETIME[1]}..."
echo "Cloning $template batch $((i + 1)) of $batches finished at ${DATETIME[1]}..." >> vmclone.log
echo "Cloning $template batch $((i + 1)) of $batches took $(( ${DATETIME[1]} - ${DATETIME[0]} )) seconds..." >> vmclone.log
echo "Cloning $template batch $((i + 1)) of $batches took $(( ${DATETIME[1]} - ${DATETIME[0]} )) seconds..."
echo "Powering on $vm_name batch $((i + 1)) of $batches..."
completedtasks=`acli vm.on $(seq -w ${startnum[$i]} ${endnum[$i]} | sed "s/^/$vm_name/" | xargs echo | tr " " ,) | grep complete | wc -l`
echo "Number of Power on tasks completed = $completedtasks" >> vmclone.log
echo "Number of Power on tasks completed = $completedtasks"
DATETIME[2]=`date +%Y%m%d%H%M%S`
echo "Powering on $vm_name batch $((i + 1)) of $batches finished at ${DATETIME[2]}..."
echo "Powering on $vm_name batch $((i + 1)) of $batches finished at ${DATETIME[2]}..." >> vmclone.log
echo "Powering on $vm_name batch $((i + 1)) of $batches took $(( ${DATETIME[2]} - ${DATETIME[1]} )) seconds..."
echo "Powering on $vm_name batch $((i + 1)) of $batches took $(( ${DATETIME[2]} - ${DATETIME[1]} )) seconds..." >> vmclone.log
done
endt=`date +%Y%m%d%H%M%S`
echo "Provisioning operations complete!"
echo "Total time to provisioning and power on all batches took $(( $endt - $startt )) seconds..."
echo "Total time to provisioning and power on all batches took $(( $endt - $startt )) seconds..." >> vmclone.log
To remove the test databases after the test is complete I used this script and executed it from one of the Nutanix Controller VM’s:
remove_sqlvms.sh:
#!/bin/bash vm_name=SQLW2K8R2AHV DATETIME=`date +%Y%m%d%H%M%S` echo "Removing $vm_name started at $DATETIME..." acli vm.off $vm_name\* DATETIME=`date +%Y%m%d%H%M%S` echo "Deleting $vm_name..." echo yes | acli vm.delete $vm_name\* DATETIME=`date +%Y%m%d%H%M%S` echo "Remove $vm_name finished at $DATETIME..."
Here is a video demonstrating what it looks like when I run the script. You can see the timing in the terminal window where I am connected into one of the Nutanix Controller VM’s running the sql_clone.sh script:
The exact same result can be achieved using the Nutanix REST API’s. In this example I have made use of the Acropolis CLI. The functions are supported equally across CLI, REST API, and within the Nutanix PRISM user interface (which leverages the Nutanix REST API’s). The entire process of provisioning the SQL Databases from template, powering them on, having them ready to run tests, from end to end, is completed in less than 90 seconds. As you can see from the video the provisioning and power on process is completed in less than 20 seconds.
It’s important to note that creating these 12 databases from template did not consume any additional storage. This is because the process leveraged the Nutanix smart clone and data avoidance features (no point duplicating data unnecessarily). Storage would only be consumed when new data is written. Storage consumption is further reduced by using Inline Compression (which also improves performance).
Each of the cloned databases is an exact replica of the master template. In these examples I didn’t customize the Windows OS for each clone, other than apply some tuning using a first boot script. Using PowerShell as part of the first boot script you could quite easily customize the OS and register with AD. For the tests I was performing I had set up the template to use a workgroup and not be part of an AD domain. Given I am rapidly creating and destroying these environments I didn’t want to have to register the VM’s and then delete the Computer Objects from AD afterwards. I wanted to treat these more as isolated test instances. This entire process can also be automated as part of a service management or cloud management platform using the Nutanix REST API.
If you want to run Microsoft apps for development and testing, or for production, no problem. The Nutanix Acropolis Hypervisor is supported by Microsoft under the SVVP program. Other application vendors have their own support statements and requirements, so please check with your application vendor or get in touch with anyone at Nutanix to check. Nutanix has many different customers running many different applications on Acropolis Hypervisor, including Oracle RAC, Oracle EBS, OBIEE, Weblogic, SQL Server, Custom Java Apps, Hadoop, Splunk etc. Nutanix has a single customer with a total of > 1500 nodes running Acropolis Hypervisor.
https://twitter.com/andreleibovici/status/614512636936503298
Final Word
By using similar methods you could create rapid test and development environments for multiple projects to use in parallel. Complete end to end application stacks can be created in seconds to minutes, all without consuming any additional space. You can destroy environments and get back to a consistent state every single time. This allows projects to rapidly iterate and provide higher quality business outcomes in much less time. Every developer or tester could have their own environment of a completely integrated end to end application stack, that they can create, test, and then destroy and recreate on demand. On large projects this could literally save months of effort and millions of dollars. All of the infrastructure management is completely integrated into the solution and provides a distributed management framework without any single points of failure. The Nutanix PRISM user interface is the Nutanix Acropolis Hypervisor management interface, and REST API endpoint. The Nutanix solution is so easy to deploy, manage, and use, that you wouldn’t even know it’s based on KVM.
—
This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – 2015 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.