One of the little known and infrequently used features of vSphere since version 4.1 is the ability to connect a USB device to an ESXi host and then mount that device to a VM, and still allow vMotion to work without any problems. This is usually used for USB dongles required for software licensing, but can be used with a number of other devices. More often these days the USB connectivity is being used from a VMware Horizon View client to connect a USB device to a desktop. But if you’re 9000 miles away from your desktop, and you’ve been asked to connect a console cable to a physical network switch in the same location and run some debug commands, how can you do that? Well I figured it out and it makes for a good story.
Back in 2010 I was helping a large company troubleshoot their virtualized SAP environment, which was experiencing instability and performance problems. One thing we noticed was that the buffers on the NIC’s were periodically overloading due to the large amount of small packets. This was on vSphere 4.0, with Windows 2003 64bit OS at the time and using VMXNET3. Unfortunately at that stage the VMXNET3 driver for Windows didn’t support increasing the send or receive buffers and as a result we had to switch over to E1000 and increase the TX and RX buffers, which resolved the problem (in addition to adding memory reservations to the VM’s). However since vSphere 4.1 it has been possible to modify the buffers in VMXNET3 to resolve these sorts of issues. I have been experiencing this myself in my home lab and have as a result modified the buffers, but it appears I may not be alone in experiencing this.
Nutanix will be once again making WebScale Waves at VMworld 2014 at booth 1535. This year we are including a big focus on enterprise applications, including SAP, Oracle, SQL Server, Exchange and Java. I will be spending time on the booth along with many other experts at Nutanix and taking 1:1 meetings, in addition to the sessions that we’ll be presenting at VMworld. My co-authors and I may also be signing copies of Virtualizing SQL Server with VMware: Doint IT Right at the Nutanix booth, so bring your copy along. To find out how you can learn more about greatly simplifying your IT infrastructure with Nutanix WebScale IT and where to party at VMworld, read on.
VMware ESXi hypervisor like many systems that implements TCP/IP uses TCP Delayed Acknowledgement to try and improve network efficiency. Essentially the idea of Delayed Ack is that a system can acknowledge every other full segment received, and must acknowledge by a certain threshold. The actual threshold can vary and be up to 500ms. While this works well for general communications and consistent streams of traffic and where full TCP segments are being sent, it can cause negative performance impacts for IP based storage systems. The main impact can be increased latency, due to the Delayed Ack Timeout, if small IO’s are being sent and received. For iSCSI on ESXi this is documented in VMware KB 1002598, however for NFS it isn’t as straight forward. I’ll take you through how to disable Delayed Ack for NFS.
I don’t like it when I get Purple Diagnostic Screens a.k.a. Purple Screen of Death or PSOD for short. Fortunately these are fairly rare. However there is one I came across just recently with a customer running vSphere 5.1 U1 and it is quite nasty. The PSOD was caused by TCP Heap exhaustion on an ESXi 5.1 U1 host. The host has the recent patches, and the usual search of the knowledge base didn’t really turn much up. The customer is running NFS, although the symptoms may not be tied only to NFS, any host based IP storage protocols (NFS or iSCSI) could be impacted. I’ll briefly tell you what we have found out, the logs to watch out for and some KB’s that will be helpful and steps you can take to prevent this from happening.
Very shortly the printing presses at VMware Press will be churning out copies of Virtualizing SQL Server with VMware: Doing IT Right, which I’ve co-authored with Michael Corey and Jeff Szastak. We had a great technical reviewer for this book in Mark Achtemichuk (Mark A – VCDX-050). There will be enough copies for everyone, and we’ll be signing them at VMworld(s), and vForum Sydney if you’d like your copy autographed (and hopefully SQL PASS events also). This is 512 pages that combines decades of experience into a book, in a single place, which is the definitive guide for virtualizing SQL Server on VMware. This project took us over 12 months to complete, and many late nights and full weekends. If you ask my family they almost didn’t see me for the entire year. This is the third book project I’ve been involved with, after being technical reviewer for VCDX Bootcamp and Virtualizing and Tuning Large-Scale Java Platforms, both also by VMware Press. Duncan Epping has done us a great honour and written a fantastic foreward for the book.
Although I have dedicated this book to my wife, Susanne, and my four sons, Sebastian, Bradley, Benjamin, and Alexander , for their ongoing support (and putting up with my absence during this project). I’ve also dedicated this book to the VMware Community.
I was also lucky to have some great sounding boards during this project in addition to my fantastic co-authors. Kasim Hansia, VMware Strategic Architect and SAP expert, Cameron Gardiner, Microsoft Senior Program Manager Azure and SQL, and Josh Odgers (VCDX-090), Nutanix Senior Solutions and Performance Architect.
Below I have included the full Table of Contents so you can take a look at what we cover and see the value that we’ve packed into this book. I hope you enjoy the book, get a signed copy, and succeed when virtualizing SQL Server.
For anyone that has been following VMware for any amount of time will know they like to keep their Beta programs very limited and usually most people don’t know what’s going on until it’s announced in public at an event. However without a wider audience involved in the use and testing of VMware vSphere in beta form there is less chance to catch bugs and the GA product quality may not be as good as what it could have been.
WebScaleIT Benefits for the Masses: Visionary Nutanix Teams with World Class Hardware and Distribution Powerhouse Dell
I have always been a big fan of the US space program and NASA as an organisation, especially for the research that they’ve contributed to. Over and above my curiosity for space travel, rocket ships, and things out in far flung places of the universe, I really admire NASA’s research and contributions to engineering, and to the inventions that have become spinoffs usable in every day life. Many of the things NASA originally developed for space, and through their space research have directly benefited us on planet earth, from medical devices, through to car tyres, to clothing for emergency services, to cordless vacuums for the home. You should check out the NASA Tech Benefits page for a list of some of these things. I can hear you asking “What does this have to do with the topic of this article?”
The answer is simple. Just as with NASA’s research and incubation of inventions that have come to benefit our lives, Google, Facebook, Microsoft and other large Web Scale companies have researched, developed and incubated technologies that can be made directly applicable to traditional enterprise IT. All that was needed was a visionary and talented organisation to package all of the inventions and lessons learned by the Web Scale companies into a bundle that was easy to consume by any sized organisation, and would meet the diverse requirements of enterprise IT.
We needed an organisation that would deliver the “always on”, “linearly scalable”, “elastic”, “pay as you grow” experience of web scale companies, at any scale, and could support a fully integrated system from end to end. An organisation that was singularly focused on removing the massive amount of unnecessary complexity from enterprise IT, on redefining customer experience, and taking complex IT and making it very simple and easy to use. This would break down silos of skills within IT and allow organisations of all sizes to concentrate more on business value creation, rather than managing infrastructure. Once we had that, we needed to take the technology to the masses, take what had been incubated and provide a few more simple options, while maintaining user experience. So lets examine briefly how and why Web Scale IT is relevant to you, and take a look at the organisations that will bring it to you, or a datacenter near you.
As you may or may not know for a few years you have been entitled to unlimited usage of Suse Linux Enterprise Server for VMware as part of your VMware vSphere Licenses and Support and Subscription. This is the same Suse Enterprise Linux that everyone was used to, just distributed for VMware under and OEM agreement. This meant that in a vSphere environment you could use as much SLES as you wanted and get all the patches and updates you need for no additional charge over and above you existing VMware SNS. If you wanted phone support on top that was a very reasonable and small fee per host. Unfortunately for those of you didn’t know about this, or for whatever reason didn’t take advantage of this then you might miss out.
While I was working on the Oracle Databases on Web Scale Tech Note for Nutanix I needed a way to quickly add disks to my Oracle RAC VM’s and set the multi-writer flag at the same time. To do this I created a PowerCLI script. It’s quite simple in what it does, but it works, and met my requirements. In the process however I ran into a VMware bug as well. So in this article I’ll explain the bug, and include my script and how it works. You can use this to add disks to an Oracle RAC node and set the multi-writer flag.