I’ve been doing work recently with SQL Server performance on the Nutanix platform. As part of this work I wanted to automate the creation of many SQL databases from template, so I could rapidly (within a couple of minutes) create and destroy them across various test iterations. This process is pretty normal for many test and development environments, although how it is achieved varies widely. This article will cover how I create the clones, and also how I’ve automated the customization and tuning of the clones after they’ve been provisioned.

As part of the process to create the SQL Databases I also needed to make sure all the drive letters were set up correctly, power management settings changed, and that the virtual NIC settings were tuned. I wrote about the driver letter problem in my article SQL Server Templates with VMware – Dude Where’s My Driver Letters. To do the clones efficiently the Nutanix platform leverages VMware’s VAAI (vSphere API for Array Integration) capabilities to do a fast copy clone. This results in significant time and space savings. A task that would otherwise take hours is completed in just a couple of minutes.

To better illustrate how this works I created a video of the process I used, which clones 12 VM’s (each 435GB, 4 vCPU, 16GB RAM), using the script included below, customizes and tunes them, and gets them ready for testing. The whole process takes around 12 minutes, but I have sped up parts of the video so you can see it happen in 4:30. This process is run on a single Nutanix 2U NX3450 block containing 4 nodes, each node has 2 x Intel Xeon E5-2650 CPU’s (2.6GHz), 256GB RAM, 2 x 10GbE Intel NIC’s, 2 x 400GB Intel S3700 SSD’s and 4 x 1TB HDD’s.

Below is the script I used to clone VM’s in bulk during the tests. This is adapted from the original by Magnus Andersson in his article Create multiple VMs from existing vSphere VM using PowerCLI.

clonevms.ps1:

#
# PowerCLI to create VMs from existing vSphere VM
# Version 1.1
# Original Author: Magnus Andersson RTS
# Updated by: Michael Webster
#
# Specify vCenter Server, vCenter Server username and vCenter Server user password
$vCenter=”YourvCenter“
$vCenterUser=”YourUser“
$vCenterUserPassword=”YourPass“
#
# Specify number of VMs you want to create
$vm_count = 20
#
#Specify where to start the VM Numbering Sequence
$start_count = 11
#
# Specify the VM you want to clone
$clone = “YourVMToClone“
#
# Specify the Customization Specification to use
$customspecification=”YourCustomizationSpec“
#
# Specify the datastore or datastore cluster placement
$ds = “YourDatastore“
#
# Specify vCenter Server Virtual Machine & Templates folder
$Folder = “YourVMFolder“
#
# Specify the vSphere Cluster
$Cluster = “YourVMCluster“
#
# Specify the VM name to the left of the – sign
$VM_prefix = “YourVMPrefix“
#
# End of user input parameters
#_______________________________________________________
#
write-host “Connecting to vCenter Server $vCenter” -foreground green
Connect-viserver $vCenter -user $vCenterUser -password $vCenterUserPassword -WarningAction 0
$start_count..(($vm_count+$start_count)-1) | foreach {
$y=”{0:D2}” -f $_
$VM_name= $VM_prefix + $y
$ESXi=Get-Cluster $Cluster | Get-VMHost -state connected | Get-Random
write-host “Creation of VM $VM_name initiated” -foreground green
#
# Create a new process for each VM that is created so they are executed in parallel, and after cloning power on the VM
#
$status=start-process powershell -NoNewWindow -ArgumentList “Add-PSSnapin VM*; Connect-viserver $vCenter -user $vCenterUser -password $vCenterUserPassword -WarningAction 0; New-VM -Name $VM_Name -VM $clone -VMHost $ESXi -Datastore $ds -Location $Folder -OSCustomizationSpec $customspecification; Start-VM $VM_Name”
}

Here is the script I used to tune the VMXNET3 Network Adapter as part of the RunOnce script during guest customization.

TuneNetworkAdapter.ps1:

# TuneNetworkAdapter.ps1
# Author: Michael Webster @vcdxnz001
Get-NetAdapterAdvancedProperty
# Using -NoRestart means the NIC isn’t restarted and changes won’t take effect till after a reboot as part of finishsetup.bat
Set-NetAdapterAdvancedProperty -DisplayName “Interrupt Moderation” -DisplayValue “Disabled” -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Jumbo Packet” -DisplayValue “Jumbo 9000” -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Rx Ring #1 Size” -DisplayValue 4096 -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Rx Ring #2 Size” -DisplayValue 4096 -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Max Tx Queues” -DisplayValue 8 -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Tx Ring Size” -DisplayValue 4096 -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Large Rx Buffers” -DisplayValue 8192 -NoRestart
Set-NetAdapterAdvancedProperty -DisplayName “Small Rx Buffers” -DisplayValue 8192 -NoRestart
Get-NetAdapterAdvancedProperty 

Here is the listing for the finishsetup.bat file, which is used as the RunOnce command to finish the setup of the guest after it has been cloned and make it ready to use.

finishsetup.bat:

@echo off
diskpart /s c:\listvolumes.txt >> c:\windows\temp\vmware-imc\diskpartlog.txt
diskpart /s c:\config_disks.txt >> c:\windows\temp\vmware-imc\diskpartlog.txt
diskpart /s c:\listvolumes.txt >> c:\windows\temp\vmware-imc\diskpartlog.txt
powercfg /l >> c:\windows\temp\vmware-imc\powercfglog.txt
powercfg /h off
powercfg /s scheme_min
powercfg /l >> c:\windows\temp\vmware-imc\powercfglog.txt
powershell -command “&{Set-ExecutionPolicy Unrestricted}”
powershell -file “c:\TuneNetworkAdapter.ps1” >> c:\windows\temp\vmware-imc\NetAdapterLog.txt
shutdown /r /f /t 0

vNIC Coalescing Scheme:

In addition to the tuning above when I was creating the template I added an advanced setting to the VM config. ethernetX.coalescingScheme = Disabled, which in this case translated to ethernet0.coalescingScheme = Disabled. I used a PowerCLI commend to set this on the VM. This disables the vNIC interrupt coalescing and as a result reduces latency. But it comes at a CPU cost. You will use additional CPU cycles. But as my tests are not CPU bound, this is an acceptable overhead to reduce the network latency.

Get-VM YourVMName | New-AdvancedSetting -Name Ethernet0.CoalescingScheme -Value “Disabled”

Final Word

With the above scripts I can clone dozens of VM’s and have them customized ready to test in < 15 minutes. The most I’ve tried at one time on my 4 node system is 40. They were all ready to use within 15 minutes and due to the space efficiency of a VAAI Clone the additional VM’s take up almost no additional storage capacity. As always comments are appreciated.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – 2015 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.