Most of the time we are sizing systems assuming that there is a slight delta in performance between workloads running native and workloads running in a virtual machine. Most of the time the assumption is the overhead is between 5% and 10% at 100% utilization on the part of virtual. However this is not always the case as many workloads can be scaled and schedule more effectively when they are virtualized. One such example is the case of a 2700+ core HPC Monte-Carlo Grid which is run on VMware vSphere for the purpose of classified missile defence simulations and research. Want to know more, then read on.

One of the sessions I’m most looking forward to attending at VMworld is VAPP5419 – High-Performance Computing (HPC) in the Virtualized Data Center. I think the topic and also the performance results are very interesting. So if you want to know the full story this is one session that you should sign up for. Edmond DeMattia will show us how this environment is used and how it is architected to achieve better than native performance. Of course performance isn’t the only benefit. Here is his session description:

The Air and Missile Defense Department’s Combat Systems Development Facility at JHU Applied Physics Laboratory relies on large-scale, Monte-Carlo simulations to perform classified combat systems performance assessments and concept studies for multiple Department of Defense sponsors. The increasing demand for modeling and simulation work drives the ever growing requirement for additional high-performance computing (HPC) capabilities, but reduced IT budgets and tribally managed infrastructures dictated that we learn how to use existing resources more efficiently. This session explains how we successfully pooled the resources of independent Linux and Windows HPC grids into a 2720-core, fully virtualized, high-performance computing platform that has allowed our engineers to achieve decreased runtimes by an order of magnitude. In addition to consolidating two dis-joined clusters for improved utilization, the ESXi abstraction layer reveals a specific use case that realizes a 2.2% performance increase over its native hardware configuration. We examine the technical and non-technical hurdles we had to overcome, such as the problems of scaling storage and network infrastructure to handle the increased number of simulations, modifying user workflows for the expanded HPC grid resources and the cultural challenges that naturally come with the sharing of computing resources. By leveraging the power of VMware vSphere, pooling disjoined HPC clusters, without compromising performance or guest OS integrity is achieved while continuing to meet strict Defense Security Services (DSS) NISPOM requirements, demonstrating VMware virtualization can reshape the path for scientific computing.

This particular HPC use case isn’t the only example of where a virtualized platform on vSphere can outperform a native implementation. WebSphere also shows some example configurations that can outperform native based on testing conducted by HP. Check out the WebSphere Performance on VMware vSphere paper.

Final Word

VMware vSphere is a great platform for running large scale mission critical applications, including HPC grid environments. The benefits are not just limited to great performance, but also greater availability and more efficient utilization and economics. As most workloads and environments are not designed to run at 100% utilization all the time any overhead associated with virtualization on VMware vSphere is imperceptible. Those workloads that are designed to run at 100% utilization as we can see from the examples above can show better than native performance, or in the worst case minimal overhead associated with virtualization (between 2 and 10%). As always your comments and feedback are appreciated and I’d love to hear of any more examples where you’ve got better than native performance on a like for like configuration in a VMware environment.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2013 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.