You may know that I don’t have an ordinary home network. Most people are happy with a simple WIFI access point. But for me, my home lab and home network is serious business. It’s where I perform all my testing for my work, including performance testing, and also beta testing of different software and hardware. Even before I joined the Nutanix R&D team I was helping companies engineer better products via the testing I did in my home lab. Early versions of the Micron PCIe SSD and Fusion-io ioDrive SCSI drivers for vSphere went through my home lab, for example, before they were released. Now I’m taking my network architecture to a whole new level. I’ve decided to implement a small scale leaf spine architecture. In this article I’ll show you how this evolution has happened, and give you some high level diagrams. This network covers Home Production, Home Test, and Home Beta Test, in addition to my work test lab.

WARNING: Don’t try this at home unless you really need to and you have a very understanding wife and family like I do.

Phase 1.0. This is my old network architecture. It served me well. But hardware eventually fails and the Cisco SGE 2000 after many years of reliable service started to lose MAC addresses every so often and require regular reboots. I also couldn’t stack any of the switches, so I had ports being blocked due to spanning tree on the port channels from the 10G switches. I use Mako Networks routers for my internet facing firewall. They are PCI-DSS certified, which is nice to know, and have a completely cloud based management interface that is very simple. Can manage a massive internet edge or merchant network from anywhere.

Phase 2.0. This is where I’m currently at. I have replaced the mix of 1G switches for a stack of 48 port Dell N2048’s. These are brand new, and I’m one of the first customers to deploy them. I chose them because they are stackable via 2 x 21Gb/s uplinks and act and are managed as one switch, and still have 2 x 10G SFP+ uplinks. The advantage of the stacking is that I can split port channels across the switches and as far as the other side knows it’s still the same switch, but I get full redundancy in the case of a switch failure or maintenance. I have tested the failover between switches and upgrades of firmware during the implementation process. These are very full featured switches. So if you’re looking for some affordable, enterprise class, fully managed, highly available, stackable switches, these would be worth checking out. You can see that I’ve moved the 10G switches to be the core. They are full L3 routing switches. They were also quite affordable compared to other 10G switches on the market and their performance and reliability has been good.

Phase 3.0. This is the next phase. I am looking to implement two additional 10G switches for the core to act as the spine. The switches support MLAG and QSFP, which should provide me sufficient bandwidth between them. This allows port channels to be split across the switches but still act as one switch, and removes the complications of spanning tree so that all the links can be active. I will have 6:1 oversubscription on the 10G leaf ports and 2:1 oversubscription on the 1G leaf ports. I will once again move the Internet and HAN equipment back to the 1Gb/s switches. The only thing missing on my Internet routers is VRRP support. One of them does load balancing of the two links, but if it goes down I have to manually change the default gateway in the core. If anyone can recommend some VDSL routers that support bridge mode, support load balancing VDSL and Ethernet WAN links, and support VRRP, I’d like to hear about it.

Although I’m starting off with MLAG’s I may experiment with ECMP as well. I’ll at least have the equipment necessary to experiment. With all of my servers connected with multiple 10G links and the type of testing I do I expect this infrastructure will get a pretty good workout. Even though it’s only small scale.

Final Word

I guess this fits squarely in the extreme home lab category. You don’t need to do this for your home lab environments and I don’t recommend it, it is a significant investment. I have taken this to the extreme because this is my business. This is what I do, and I want to be globally competitive and productive, even while living at the bottom of the world in Auckland, New Zealand, working from home. I design systems for large scale global organizations, I need something that functionally looks like what they might have and performs.

There are many ways you could design a scalable network architecture for your home or your business. There are many vendors that would allow you to deploy a leaf-spine architecture, which allows you to scale simply and linearly without introducing a lot of bottlenecks. All the major vendors can do this, and they all have their various differentiators. If you wanted to set up something quite functional, cheap and simple you could easily do it with virtual appliances running on a desktop or laptop. But as I do real performance testing at home I need enterprise grade equipment.

I chose Dell for my home lab due to their reliability, performance, enterprise grade features, and support (24/7/365 with 4hr response), and because it was something I could see value in and justify to myself as an affordable investment. I’ve had a long relationship with Dell before I joined Nutanix and they were able to give good deal on their networking equipment. Standarization makes things a lot easier and less likely to introduce incompatibilities or configuration errors. Their network equipment has served me well, as did the mix I had before. I’ve now had my 10G switches running for a couple of years. Cost per port and features of the Dell networking equipment is very competitive, and quite affordable for small and medium businesses.

As always your feedback and comments are appreciated.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – 2014 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.