| Unique Visitors |
Powered By 
| Unique Visitors |
Although my Monster VM panel was in the top 10 sessions of VMworld 2013 and we did a Monster VM and Business Critical Apps panel for TAM day this year neither session will be included at VMworld in the USA or Europe. But not to worry. There is plenty of great content at VMworld for everyone to enjoy and I’ll be there to talk about Monster VM’s on vSphere as always, but mostly at the Nutanix Booth #1535. This doesn’t mean you have to miss out on all the Monster VM goodness though, you can grab it all online right now, for free, thanks to VMware opening up the session catalog to some great sessions online through VMworld TV. So below I include two great panel discussions from VMworld 2013 that you can review now, and get a better understanding of how to virtualize critical apps and Monster VM’s.
This first video is from TAM day and includes myself and a number of leading experts discussing virtualization of business critical apps and taking questions from the audience. The topics covered are broad from Microsoft apps such as SQL, Exchange, AD to Unix applications and migrations from Unix systems to vSphere, Oracle, SAP and Java.
This second video is from my Monster VM Software Defined Datacenter Design Panel, which was one of the top 10 sessions for VMworld 2013. We cover sizing, tuning, and scaling, as well as answering all the questions from the audience. When you get 5 VCDX in the room talking about performance and monster VM’s at the same time it’s got to be good.
Final Word
Even though we don’t have a Monster VM panel at VMworld this year there is no reason you should miss out. Come and visit the Nutanix Booth #1535 and I’d be happy to discuss everything Monster VM related, and also how the Nutanix platform can make it uncompromisingly simple. I will have copies of the book Virtualizing SQL Server with VMware: Doing IT Right (VMware Press), that I co-authored with Michael Corey and Jeff Szastak that we’ll be signing. If you can’t make VMworld in the USA and you’d like to go to VMworld in Barcelona then you might be in luck. VMTurbo has started a competition and will be giving away free tickets to VMworld Europe. VMTurbo started the vendor trend to give away free tickets to the VMworld events and I think it’s a great thing. As always your feedback and comments appreciated.
—
This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – 2014 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.
At VMworld in 2013, which had 25,000 attendees, it was fantastic that one of my sessions got into the top 10 sessions of the entire event. With big thanks to my co-speakers Andrew Mitchell, Frank Denneman, Mark Achtemichuk, Emad Benjamin and Mostafa Khalil on the VAPP4679 – Software Defined Datacenter Design Panel for Monster VM’s. The audience agreed we took the Technology to the Limits for High Performance, High Utilization Workloads. We had some excellent questions and feedback during this event. This year I am upsizing the Monsters and brining you even more of what you asked for. Performance data, design and architecture guidance, Monster VM sink hole avoidance strategies. The Monster VM panel this year is super sized and I’m lucky to be joined in this session with Frank Denneman, Mostafa Kahlil, Josh Odgers. If you liked what you saw last year, which you can watch the recording of below, then you’ll love what we have for you this year. We’re certainly aiming to be in the top 10 sessions again of VMworld 2014 (assuming we are selected of course). Your support really helps.
VMworld 2013 VAPP4679 – Software-Defined Datacenter Design Panel for Monster VM’s
You can see all of the Top 10 Sessions of VMworld 2013 here.
VMworld 2014 Monster VM Sessions:
Here are three Monster VM focused sessions that I hope you’ll like and vote for. You can vote for your favourite sessions on the VMworld Session Voting web site.
Session 1302 – Mastering Monster VM Performance in Your Software-Defined Datacenter
Top 10 Session at VMworld 2013, now even more of a Super-Sized Monster for 2014. Master your Monster VM Performance. Master your Software-Defined Datacenter. Designing and architecting a Software-Defined Datacenter that will host Monster VM’s requires a different approach and different considerations to smaller VM’s to ensure you get the best benefits from the platform. Mostafa Khalil (VCDX-002), Frank Denneman (VCDX-029), Michael Webster (VCDX-066), and Josh Odgers (VCDX-090) will lead a panel discussion the key best practices that will allow you to master design and performance of many high utilization, high performance Monster VM’s. The panel will share real world performance data and answer your questions on topics such as host and cluster design and sizing, VM sizing, performance tuning, CPU scheduling, memory management, resource management, storage design and much more. Come prepared with your toughest Monster VM questions and a willingness to share you experiences.
Session 1729 – 47 Things to Know About Database Virtualization
This advanced technical, no fluff session will cover 47 unique items that individuals looking to virtualize their databases should know. The session will provide items that apply for SQL and Oracle databases running on vSphere. Presenting the session will be VMworld veterans, co-authors of Virtualizing SQL Server on VMware: Doing IT Right (VMware Press), and database virtualization experts Michael Corey, Michael Webster (VCDX 066), and Jeff Szastak.
Topics covered will be, but not limited to, physical CPU virtual CPU, NUMA, physical RAM, virtual RAM, PVSCI tuning, storage (lots of storage), physical server configuration, ESXi configurations, vSphere Cluster design, OS tuning, database specific tuning, VMXNET tuning, jumbo frames, and more!
1376 Temenos T24 Migrated from AIX to vSphere 5.1 and SQL Server 2012
This sessions would be one I recommend. Termenos is a core banking system used throughout the world. See how Rene helped Bank AlBilad virtualize their core banking platform on VMware vSphere with a Mosnter VM SQL Database. I was fortunate to be able to work with Rene on this project specifically with SQL Server and vSphere tuning.
Non-Monster VM sessions I’m involved with:
Session 1600 – Art of IT Infrastructure Design, The Way of the VCDX – PanelNon-Monster VM Sessions:
This session covers IT Infrastructure Design Methodology as practiced by both VCAP Design and VCDX certified architects. The session covers the what, how, and why of the methodology. Examples are provided and the panel discusses their experience as panelists reviewing and validating hundreds of designs. Case studies are presented for designs with customers. One example demonstrates design success. The other example demonstrates design failure. The panel discusses and contrasts the approaches of both.
Session 2631 – The Way of the VCDX, Testing Architects on IT Infrastructure Design
Final Word
Please vote for all your favourite session. It helps the selection committee make sure they have enough of the content that you want. Vote Here, Vote Now!
—
This post appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2014 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.
I’ve recently been testing some Monster VM’s in my lab and setting up templates to provision these Monster VM’s more rapidly. Along the way I’ve run into a few things I wasn’t aware of that could be a potential barrier to Monster VM’s in some ways, and some interesting behaviour during large storage migrations. When you start provisioning Monster VM’s more often and move them around you are quite likely to run into the same problems. So hopefully this article will help you work around the problems I hit and avoid them until there is a more permanent solution. I also hope that our friends at VMware are aware of this behaviour and work towards addressing it in an upcoming release so that Apps can continue to love vSphere as much as they do today, even when the scale increases and they become the norm. I would like to thank Duncan Epping, Andrew Mitchell and Frank Denneman for their help with understanding this problem and it’s possible solutions.
Firstly let me qualify what I mean by a Monster VM. When I say Monster VM in this case I mean a VM with 8TB of disk (4 vSCSI Controllers and 17 VMDK’s), 16 vCPU’s and 192GB RAM. So it’s pretty big, but not massive compared to the maximums that vSphere 5.x can support. But this was big enough for the experiments I was going to run with Microsoft Exchange 2010, Exchange 2013 and Oracle 11g R2.
The Problem and Why it’s a Barrier to Monster VM’s
So what’s the problem? Well the problem is I was creating templates for these Monster VM’s so I can provision them rapidly after my experiments. These templates are thin provisioned so I don’t waste unnecessary space. Also the systems I deployed from these templates and was running the experiments on were also thin provisioned. Where the problem comes in is I had put these VM’s on my bulk NAS storage until I need them. This was so they don’t take up space on my higher more valuable tiers of storage (See my lab environment). I hear you saying, so what could possibly cause a problem with that?
Well that is the interesting part. It all comes down to a thing called a Data Mover. A Data Mover is the component within VMware vSphere that moves data from place to place (funny that) when doing operations such as Storage vMotion and other types of storage migrations (clones). In vSphere today (4.1 and up) there are three different Data Movers.
The three Data Movers are FSDM, which is the most simple, most compatible and slow data mover (highest level in the IO stack too), FS3DM, more advanced and faster, and FS3DM Hardware Accelerated (offloads to the array). I will not go into all the details here as I think the best way to understand these is to read Duncan and Frank’s Book – vSphere 5.1 Clustering Deep Dive, and Frank’s article VAAI hw offload and Storage vMotion between two Storage Arrays.
In my case I was migrating between arrays, going from my NAS, which is NFS, to either another NAS (NFS filesystem), or a VMFS5 datastore on an iSCSI array. My iSCSI Array (HP StorVirtual) and one of my NFS systems (Nutanix NX-3450) have VAAI enabled, my other NAS (QNAP 869 Pro, QNAP 469) don’t have VAAI yet. This is where I hit a little snag. Going between arrays regardless of their VAAI capability, won’t use FS3DM Hardware Accelerated Data Mover, that’s only used within the same array. But going between a NAS and VMFS5, or from one NAS to another NAS won’t even use FS3DM. It will only use FSDM, the most compatible, but also the slowest and simplest of the Data Mover species. Why is this so important I hear you say? Let me explain.
FSDM will read every block of a VM, even if it has never been written to. You read that correctly, even if the block has never existed, it will be read and transferred. This is very bad news if you have a Monster VM or Monster VM Template using Thin Provisioning. This effects every vendor using NFS with VMware vSphere. One of the reasons it probably isn’t that well known or noticeable is that most VM’s have very small storage footprints, relatively speaking, especially when they are initial provisioned. Large production VM’s also don’t move around all that much.
So in my case one of the VM’s had 8TB allocated, 5.65TB in use, another VM had 5TB allocated and only 15GB in use. Guess what it did. It copied over all the real data, and then for all the VMDK’s that had zero bytes physically written on disk it proceeded to copy TB’s worth of zeros for every block that had never yet been written to. The storage migration took 3 days to complete! In case you were wondering I have a 10G network but the QNAP NAS is dual 1G connected.
In contrast to this, when I migrated or cloned the template VM with 15GB used and 5TB allocated from VMFS5 to VMFS5 and it used the FS3DM data mover (I did this across arrays) it took all of 24 minutes. 24 minutes (VMFS5-VMFS5 across arrays) vs 3 days (NAS-VMFS5, NAS-NAS). You see my problem.
The Workaround or Temporary Solutions
So I have considered a number of solutions or workarounds.
1. I could mount an iSCSI LUN on my NAS and use VMFS5 to store my templates and make sure their destination is also VMFS5. This really isn’t ideal, as there are a lot of advantages to NFS, simplicity being one of them. Still wouldn’t help if I needed to deploy the template on an NFS datastore.
2. Only put the OS disks on my templates and leave off all the other VMDK’s and SCSI controllers until after the VM’s are provisioned. This isn’t ideal either as I’ll have to re-provision all the apps and data each time I want to re-create a VM and run a test from a known good state.
3. I could provision and set up a VM on each of the storage devices I’m going to use for my experiments. This option is also not ideal, as I’ll have to build these templates on each storage device. But at least being thin provisioned they won’t take up much space initially, at least for the templates.
4. Instead of using Storage vMotion I could use Backup and Restore, with say VMware VDP and vADP (vSphere API for Data Protection). This is potentially a good option, and might be quite fast.
5. I could set up vCloud Director and use Fast Provisioning. Provided I set up the template and use the same underlying datastore for all the copies this would work fine. This is one of the advantages of NFS, you get one huge volume, but VM level granularity. If I wanted to provision a copy to another datastore though, it would probably be hit by the same problem.
6. It’s NFS. I could just copy the files using the filesystem and not using a Storage vMotion or cloning. This is fine in my lab environment where the VM’s will likely be powered off when I want to move them about. But in a production environment where the VM’s need to remain powered on, this is not going to work.
In a production environment if I was looking to migrate a Monster VM that was running a database or Exchange or similar I would probably set up a new instance on the destination and use the application level utilities to migrate the data and switch over. Your requirements will vary, so evaluate the best options for your situation.
Putting this in perspective
In order for you to come experience this you need to be migrating cloning or deploying VM templates that have a source or destination on different storage devices, backed by an NFS Datastore, have Thin Provisioned Storage, and have a large storage allocation 1TB+. So right now this might not impact many. But the numbers will grow over time. Especially when you consider that many Cloud environments, especially those using vCloud Director, are backed by NFS storage, as it’s more efficient for Cloud. Although many environments may still be using block storage right now for the majority of their workloads, I think in the future that will change. Many systems are now running on NFS, and many vSphere environments are now backed by NFS Datastores. This will increase in the future.
Final Word
I think the real solution is that the process VMware uses for migrating large VM’s between arrays where a NAS or NFS is involved, especially those that might be thin provisioned, needs to change. The FSDM Data Mover doesn’t cut it, not for Monster VM’s. Copying TB’s worth of zeros that have never been written to is of absolutely no value. It can be done much smarter. Perhaps this might come in the form of some new VAAI for NAS Primitive. Perhaps VMware just invents a new Data Mover. They have many smart people and are committed to Monster VM’s and running these types of workloads in a Software Defined Datacenter or Hybrid Cloud, so I’m sure in due course a viable long term solution will become available. Until then I’ve probably only thought of a subset of the options, so I’d like to hear your thoughts on this.
—
This post appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2013 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.