I say "VMware" homelab, but obviously this could be any virtualization solution, but I have built this specifically for my VMware learning, so at the moment, that is my primary focus.
I started building my first homelab during COVID (around November 2020). Prior to that the only kit i really had at home (other than a laptop) was my Synology NAS which i use for storage. 3 and a half years later, i'm in the process of building the 3rd version of my homelab, and given it takes a lot of time trying to figure out what hardware I need or how to resolve issues and limitations with each rebuild, I thought it would be useful to document my journey so far, in the hope it might help others.
Why build a homelab?
If you want to develop your VMware skills, building a homelab is probably the best way to do it. Not only will you learn how to actually put together a homelab environment, but once you have a homelab, you can build and deploy all sorts of cool stuff. From a single ESXi host to an all singing, all dancing full VCF environment, or anything else that takes your fancy.
What kind of homelab you build and what hardware you choose, really depends what you want to do and how far you want to go. If this is your first time building a homelab, I would probably advise keeping things simple and starting out with something easy to build and setup (such as a NUC or similar). You can build up your lab, and gradually figure out what you want to do and how you want to upgrade as you go.
Multiple Iterations
If there's one thing i've learned about homelabs, its that you're always hitting a limitation somewhere, or thinking of better ways of doing things. For this reason, i'm in the process of building version 3 of my Homelab. I have invested in some new hardware for the compute side (more on that below), and i'm reworking the storage and networking side of things as I need to significantly improve the storage performance.
Version 1 of my homelab was very straightforward. This involved buying a NUC, and some decent components (which i've documented below), and installing a vSphere 6,7 environment with a couple of nested ESXi hosts and a vCenter. I used this for deployment experience, and also for upgrade testing for vSphere 7.0.
DNS is an essential part of any homelab, and I covered this by building an Ubuntu VM on the NUC and installing BIND. I used a 192.168.68.x range that was separate to what my router was using, so I had total control of that subnet for my homelab. Ubuntu isn't particularly difficult to learn by the way. I'm not a BIND expert by any means, but 2 of the organisations i worked at previously used BIND for DNS, so i was familiar with its existence, and of course its entirely open source and really quite easy to setup. There are plenty of guides to walk you through the process, and i highly recommend giving it a go. This setup worked well for a while, and in fact, I passed my VCP, with the experience gained from this setup.
Version 2 followed about a year later. I already had a pretty decent Synology NAS which i was only using for a filestore, but I knew it supported iSCSI as well. So version 2 of my homelab involved rebuilding to vSphere 7 and using the Synology to provide iSCSI storage to my homelab. Again this worked well to a point, but I was hitting the limit with the 32GB of RAM on the NUC and I was experiencing storage performance issues due to the spinning disks in my NAS obviously not being up to the task. Connectivity was also over Wifi mesh which wasn't great either.
I also needed to progress to deploying vSAN and VCF in my Homelab, which was not possible in my V2 setup, so Version 3 Homelab was required.
Version 3 (current and in progress)
I've listed the issues I need to fix for version 3 of my homelab (and their solutions in brackets) below:
Lots more RAM (Yep, need a new machine for that)
Faster VM storage (Ditch spinning disk and move to SSD/NVMe)
Better connected storage (Move VMs from NAS to local disk)
Add VLAN capability (buy a managed switch)
Eliminate Wifi (Move Synology to the above mentioned switch. Note that although I plan to move VM storage to local disk, I still plan to use the Synology for NFS)
Repurpose the existing NUC as a DR site
Move DNS off the homelab so it isn't a VM or a circular dependency. The Synology is capable of running a DNS server, so we'll use that for the version 3 homelab instead.
For compute, I could have upgraded my NUC to 64GB, or even bought another NUC, but planning ahead, I knew this still wouldn't be enough for VCF, and buying another NUC wasn't cost effective. I needed something a lot more capable, the question was, what? I read numerous blog posts to see what others were doing and found homelabbers using all sorts of hardware. Some purpose built, some whatever old kit they could get their hands on, but basically a wide array of all sorts of solutions, and nothing was really jumping out at me.
I focussed on what would give me lots of RAM whilst not be ridiculously expensive, and after much reading around, I eventually settled on the Dell Precision T7910 workstation.
The Dell Precision T7910
The main reason i chose the Dell T7910 was because it can take up to 1TB of RAM. As soon as i read that, I knew this was my unit of choice! The T7910 was released in 2014, so is over 10 years old now (at the time of writing), however don't let this put you off, as it is still an extremely capable machine, and can be found on eBay for a fairly reasonable price.
The T7910 is a dual socket machine, and the maximum 1TB RAM can only be achieved when both sockets are populated, and when all modules are 64GB LRDIMMs. Purchased new, the required 64GB DDR4 ECC LRDIMMs are not cheap, but I managed to find 2 "used" modules on ebay for around £50 each. Coupled with an additional 2x32GB modules, I was soon up and running with 192GB RAM (Already 6 times more than i had in the NUC!!)
The T7910 can take either v3 or v4 Intel Xeon E5 processors, 4 x HDD/SSDs plus 4 more with the optional expander, and it has a generous 6 x Gen 3 PCIe slots. On further reading around, I found you can buy 4 slot NVMe adapters that install into a PCIe slot, giving even more storage options and blistering fast performance.
In short, the T7910 has met all of my expectations so far, and in some areas (such as its extensive upgradeability options) it has exceeded them. For about 1.5 times the cost of the NUC, I have about 6 times the compute power, with plenty more capacity to go.
Intelligent Servers UK
If you are interested in buying a T7910 in the UK, I would highly recommend Intelligent Servers who I purchased my machine from.
I have no affiliation with the company whatsoever, but they provided a really fast and friendly service, and I was impressed that they called me a few weeks later to see how i was getting on with the machine. I can honestly say i've never had that from any seller before in over 20 years of ebaying! Of course there's an element of follow on business, but it was a very friendly chat and not pushy at all. Their primary concern was making sure I was happy with my purchase, and the answer was yes, very much so!
Links to Intelligent Servers can be found below:
Ebay Store - https://www.ebay.co.uk/str/intelligentservers
Main Website - https://intelligentservers.co.uk
Homelab Specifications and costs
This is the first time i've written down all the costs spent so far (from the Synology right up to and including the version 3 hardware i've just purchased), and i was surprised at how much it all added up to!! The total cost spent so far is £1,699.12 (ouch!). However, it should be noted that i'm writing this post in March 2024, but I have included the Synology which i purchased way back in Jan 2016 and the NUC in Nov 2020, so to be fair, this has been a gradual process where the cost has been spread out over a long period of time. Also worth bearing in mind, if this helps support your certification and career progression, the payback should make it more than worth it.
Incidentally i've included the NUC spec below to give you an idea of the build i used for my first V1 homelab. I'm planning to repurpose it as my new DR site, so it remains a valid part of the version 3 build.
Note the parts in green have been purchased and/or installed, but have not been brought into use just yet. As you can see, i'm acquiring parts towards the improved storage and networking capability, which were 2 key goals for my version 3 build.
Dell Workstation (Cost including additional components £728.77)
Dell Precision Workstation Tower 7910 (aka T7910) (used) - £415
2 x Intel Xeon E5-2660v3 (2.6GHz 10 core CPU) - included
2 x 16GB DDR4 RAM - included (but removed for larger capacity modules)
NVIDIA NVS 315 DMS-59 Dell P/N 0MD7CH PCIe Graphics Card (included)
Hynix HFS256G32MND 256GB SSD drive - included
2 x 64GB Hynix DDR4 ECC LRDIMM P/N HMAA8GL7AMR4N (used) - £99.98
2 x 32GB Samsung DDR4 ECC LRDIMMs (used) - £45.00
Crucial BX500 2TB SATA SSD (used) - £78.80
Kingston SA400S37 240GB SATA SSD - FREE
ASUS Hyper M.2 x16 Card v2 4 x M.2 Socket (new) - £44.99
WD SN720 SED NVMe M.2 1TB SSD - £45.00 (used)
Intel NUC (£563.97) - Purchased new (Nov 2020 prices)
Intel NUC 10 i5 10210U (P/N: BXNUC10I5FNH3) - £316
Samsung 970 EVO Pus 500GB PCIe NVMe M.2 SSD - £69.99
Crucial MX500 1TB CT1000MX500SSD1 SSD - £79.99
Crucial CT32G4SFD8266 32 GB DDR4, 2666 MHz, SODIMM - £97.99
Synology NAS Diskstation (£382.95) - Jan 2016 prices
Synology Diskstation DS416J £235.86
3 x 3TB Western Digital WD30EZRX Drives (used) - £147.09
Networking (£23.50)
Netgear GS308E 8 Port Managed Switch - £23.50
At the time of writing (8 March 2024) I have only just bought the parts highlighted in green, but I have included them as part of the specification and build costs, as i will be bringing these into use over the next few weeks.
Hopefully you found the post useful. Any questions, feel free to leave a comment!
Mark
Comments