The Home IT Lab is just about finished and boy does it feel good after almost three months of scavenging for parts, buying certifiable junk off eBay, crawling through a dusty attic to run cable, and constructing something robust & redundant(ish) enough that I could use it as an experimental tech lab while providing four, maybe 5, 9s of reliability for my most important users, the fam.
At times it felt like one step forward and two steps back, but this is the final configuration. Yes, I’m done with the bare metal provisioning:
Not bad eh? Sure the cabling could be dressed up a bit, but overall, I’m pretty happy with how it came together.
So, here’s the setup (So far):
Already I’ve learned so much that’s applicable at work and in the lab.
The first thing I’ve learned: ZFS is awesome, and the open source guys and companies who labor over these amazing storage network operating systems are heros in my book.
Over the last 30 days, I’ve tested FreeNAS, OpenFiler, NAS4Free, and NexentaStor. Initially I didn’t want to sacrifice an entire physical host for simple storage duties, but building a Scaled Out File Server running Windows Server (ie the Windows SAN idea that’s legitimately interesting yet completely frightening at he same time) wasn’t a possibility here, so I had to sacrifice.
Now let’s be honest: what kind of performance would you expect out of a bunch of 2009-era AMD Semprons, some REALTEK Gbit adapters, consumer grade SSD & HDD, and Cat5e cabling pulled through this attic over the course of a few weekends?
Well, I think I’m hitting this stack about as hard as it can be hit without enabling jumbo frames or going RAID 0. FreeNAS and the others, while feature rich and sporting great cacti reporting, fell over at times during some really harsh and cruel SQLIO benchmark runs.
But NexentaSTOR didn’t. Here’s what the Sempron + 4xHDD, 1 SSD cache, 1xSSD logs and 16GB RAM turned in last night:
Yeah, I know, kind of unbelievable. Like, so unbelievable I had to run it a few times to make sure, and even now, I’m worried something isn’t quite right with the stats I’ve gathered. I mean I don’t know squat about L2ARC caching (except that it costs a lot of RAM) and logs, but I do know quite a bit about optimizing switch, LACP, and Hyper-V.
This is what it looked like from the Nexenta dashboard and one of my vEthernet NICs. Notice the read rate on Drive E/Disk 13:
Perfmon recorded this on Node1, comprised of a 3x1GbE team, with two Intel server NICs and a RealTek:
Fun times indeed.
One of my favorite virtualization bloggers, Everything Should Be Virtual.com, also found Nexenta to be a great fit for his home lab.
So the lab is built now. And it’s solid and robust enough for my needs. I have Hyper-V virtual machine failover between Node1 (office) and Node2 (Garage), I have passed cluster validation, thrown a ton of packets at the NexentaStor, and nothing caught fire, with the Nexenta giving me some CPU warnings but still able to write & read more data than a 1GbE NIC would have allowed otherwise.
I still have to get the vSphere host up. 5.5 doesn’t like RealTek nics, and I’m cheap, but I’ll figure something out. And if I don’t, Xen or KVM will go on that box.
Next up: I have about 80 days left on these Windows trials. I’m going to take full advantage and spin up a System Center instance. Then I’m going to play with Splunk and start again on building an OwnCloud VM to capture, index and app-ify all my personal family photos, music, videos, etc.
Also thinking of making the SG-300 my default gateway, or getting around to finally using a Vyatta VM Router or pfSense VM as my gateway. Following that: ipv6 experiments with Pertino, Hurricane Electric and more.
Good times!
Wilson.tech 
