Many companies are starting to run Jumbo Frames across their network to improve performance, especially as 10Gbe infrastructures become the norm. When you’re making a change such as implementing Jumbo Frames it’s important to evaluate what possible implications it might have. One such implication that I wasn’t aware of till today is a possible problem with Jumbo Frames when used on your VMKernel Management Network with vSphere 5.0 and vSphere HA.
Duncan Epping at Yellow-Bricks.com posted an article describing the situation titled “No Jumbo Frames on your Management Network” and pointing to a VMware KB article 2006729. The KB article only mentions Jumbo Frames in a very small not right at the very bottom. I would strongly recommend you read Duncan’s post for more information.
I personally haven’t come across this problem even though I run Jumbo Frames across my entire network in my lab, including on the VMKernel Management Network used for HA. However I have run into other problems with Jumbo Frames support in the VMware VMXNET3 Driver when using the VMware Tools version from ESXi 5.0. My advice is that you should have a thorough test plan whenever you are making a significant infrastructure change, and evaluate all the pro’s and con’s. There are significant advantages when running Jumbo Frames on 10G networks, and even advantages on 1G networks, although not the same magnitude.
Update [20120313]: It turns out that all of the problems reported to date have been related to incorrect configuration. So it appears this is not a problem after all.
This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.
Michael, to re-inforce your points I have finsihed converging my production environment into FlexPod architecture (UCS – ESXi 5.1- Nexus – Netapp – Cisco 3750-Lan) I am using jumbo frames on all my vmnics and i havent seen any issues at all, before migrating of course i was thorough on my research om jumbo frames and had a long test plan. I use NFS, and iSCSI for storage on the Netapp and also iSCSI boot on the esxi UCS blades. I haven’t seen any impact in performance only the expected gain in troughput and latency reduction. Jumbo frames are not enabled on my LAN swicthes but are on the Service template nics, Nexus , netapp interfaces, esxi vmnics (including vmkernel ports) …all is working well. No CPU increase in any of the networking devices was noticed either in any of the devices.