In which the Hyper-V guy plays with ESXi 5.5

All the sweat equity, money, and time I’ve put into the home lab is finally paying off at the Agnostic Computing.com HQ.

In fact, it’s been great: satisfying and pleasing little green health icons are everywhere, I read with satisfaction the validated Microsoft cluster configuration reports without any warnings at all, and the failover testing? Let’s just say I can remove “ish” from the end of the word “redudant.” This stack is as solid as it’s going to get on my low budget, single-psu setup designed to draw fewer than 5 amps and less than 500 watts (I’m at about ~325w & 3.5 amps more or less)

But standing up Hyper-V clusters on consumer-grade hardware isn’t exactly expanding my portfolio, even if all my storage is parked in a (new to me) ZFS box.  So last weekend it was time to tackle Hyper-V’s nemesis: VMWare’s market-dominating ESXi 5.5, which I’ve got running on a stable 2-core Athlon II box, 12GB of RAM, and an Intel 2x1GbE NIC.

For a Hyper-V guy who hasn’t touched ESXi since probably 2011, building out the ESXi box involved some trips down memory lane.

A memory lane called Pain Street.

The last time I worked in ESXi on anything meaningful was during an eight month span during 2011 in which my colleagues and I were charged with replacing ESXi with Hyper-V 2.0, baked into the just-released 2008 R2 edition.

We had Hyper-V 2.0, a few brand-new PowerEdge servers with quad-Nehalem CPUs, something like 512GB of RAM, a FAS 2210, System Center Virtual Machine Manager, 2007 Edition, and a brand new file system-like layer on top of NTFS called Cluster Shared Volumes.

Oh, and a handful of V2V tools & .vmdk to .vhd conversion scripts with which we planned to stick it to VMWare.

I mentioned that this was a painful time in my life, right?

I’ll save the Hyper-V war stories and show you my scars (Hyper-V virtual switch ARP storms, oh my!) another time, but here’s what I learned from that experience: Hyper-V 2.0, was in all ways inferior to ESXi when it debuted in Server 2008 R2. And not just a little inferior. No, we are talking NBA vs 8th Grade Boys Basketball team scale inferiority.

It was half-baked, not entirely thought out, difficult to scale, prone to random failures, hard to backup (even risky…sometimes the CSVs would just drop off when the IO was supposed to be redirected to another host), and the virtual drivers written by Microsoft for Microsoft Hyper_v virtual machines running on Microsoft virtual synthetic NICs weren’t stable. It was a hypervisor that made you pound your keyboard, sit back in your chair, scratch your head and ask, “Has anyone at Microsoft ever tried to use this thing?”

And you couldn’t team it and expect Microsoft support. I had to delay my love letter to LACP for years because of that.

Even so, I loved Hyper-V 2.0. Wore the admin hat like a badge of honor. Proud and boastful of the things I could make Hyper-V 2.0 do in the face of so much adversity, so much genetic disadvantage. Yeah the other guys had Ferraris tuned up by Enzo himself and all I had was a leaky Fiesta with a suspect axle, but that Fiesta could, in the right hands, make it across the finish line.

We, we happy few, we band of brothers, who persisted in our IT careers through the days of Hyper-V 2.0 and even excelled.

All that to say that the hey-day of VMWare, ESXi, the Nexus 1000v, and now VSAN have kind of passed me by. Just can’t seem to get exposed to it, to sink my teeth into that whole wondrous stack. It’s expensive.

But it’s been alright with me because in the same span I’ve adopted Hyper-V 3.0 with relish and become convinced that we Microsofties finally had a Hypervisor worthy of respect. “Feature Parity” is a term that’s been bandied about, and with 2012 R2, it got even better. EMC, parent company of VMWare, even called SMB 3.0 “the future of storage.” Haha, take that NFS!

So has it?

It’s not easy for me to admit this but while I like Hyper-V much more in some areas and feel like it can scale and serve any enterprise well, I have to admit after playing with ESXi at home, Hyper-V still has deficits purely from a Hyper-visor perspective (System Center is a different animal).

Deficits other virtualization bloggers are eager to demonstrate, with barely-concealed glee. Take Mike Laverick, a sharp ESXi guy, for instance. This February readers of his blog have been treated to post after post In Which the ESXi Guy Plays with Hyper-V 3.0.

I’m always up for a good tech debate, but after devouring his posts, letting them sink in, I got nothin’ except a few meek responses and maybe some envy.

He concludes at the bottom of this great screenshot-by-screenshot comparison:

I guess to be fair – taken individually this lack of hotness of the Gen2 Windows 2012 Hyper-VM might not be a deal breaker for some. For me personally, they collectively add up big pain in the rear, especially if you coming off the back of virtualization product like VMware vSphere that does have them. For me the whole point of virtualization is it liberates us from the limitations of the physical world. What’s the point of software-defined-virtual-machines, when it feels more like the hardware-defined-physical-machines….

’tis true in some respects. I have long wanted to stop mapping LUNs directly from the SAN, through the Hyper-V switch to a virtual machine, but it was not possible to resize .vhdx drives on a live VM until October 2013, when R2 was released. And even now in R2, it’s not as simple or more importantly -reliable- enough to depend on in production, at least not compared to resizing an RDM in a NetApp or Nimble or even my ZFS array.

I will offer some resistance in the following two areas though.

Hyper-V runs on whatever piece of junk you throw at it. That’s interesting news if you’re a value-oriented enterprise, and really great news if you’re building a home lab or trying to learn the trade. VMWare, in contrast, won’t even install without supported NICs…the cheap realtek in your Asus? Not supported. The Ferrari metaphor is apt: You’ve got to shell out some bucks for the High Octane stuff before you can stand-up ESXi in a meaningful way.

Second observation is that I’m not comprehending the switching model very well. I was really excited to see Cisco Discovery Protocol just work on mouse-hover with zero configuration, but this 1:1 stuff feels archaic, devoid of the abstract fabric goodness:

What am I missing here?

On my ESXi box, I’ve got two Intel GigE adapters. I have the option to make them active/passive (cool), team them, but I’m not seeing the same converged fabric concept that’s liberated me in Hyper-V 3.0 from, guess what, worrying about hardware.

The three NICs on my Hyper-V host, for instance, are joined in an LACP team, which then is used to build a true & advanced virtual switch for both the host & the guests. And an LACP-capable switch is not a requirement here; I could use the dumb switch in my rack and have the same fault-tolerant (though lower performing) converged team.

Some very simple powershell lines later, and you’ve got vethernets  on the management OS tagged with the appropriate VLAN.

All ports on the physical Cisco switch? Trunked.

Freedom.

I know I’m missing something here…PowerCLI? I’ll be testing that tonight.