I'm building a new EPYC Rome node for my home data center, trying to figure out if OmniOS can run everything I want. The last component just arrived and the machine is memtesting right now.

Over the years I've ran mostly ESXi and I've also tried to love (but didn't) Proxmox. My new node has a twin sister with a Skylake Xeon, running Fedora. If everything goes to plan the twin can be migrated to OmniOS later too.

Some specs:
- Inter-Tech 4U-4408 storage chassis
- Supermicro H11SSL-i motherboard
- AMD EPYC 7302P CPU (16 cores)
- 8x 16 GB DDR4 3200 ECC RDIMM (Samsung)
- 1x 2 TB M.2 NVMe WD SN700
- 2x 16 TB HDD Toshiba MG09

This motherboard is a little funny. You get two SFF-8643 connectors, but there's no SAS chip, so you can only break them out to SATA. Suits me fine for this build, the chassis backplanes happily take SFF-8643 too. The H11SSL-i motherboard is alright. EPYC Rome also fits a H12SSL-i which would get me PCIe 4.0, but the price for the motherboard + CPU combo would have doubled and I don't really need it for this build.

The Inter-Tech (German brand) chassis are pretty nice for affordable DIY builds. 4U is sweet because you can use regular ATX PSU's and fit heat sinks with fans that don't sound like jet engines. The heat sink I have now is a "CooNong" (? bless me) that I got with the motherboard. Looks like a clone of the same style that Supermicro sells. It came with the most crappy fan humanity ever laid eyes on so I promptly fitted a Noctua like the rest of the fans. As usual, motherboard warns about fan speeds being too low, expecting high RPM data center fans, but temps are otherwise fine. At least there's no audible warning for the fan speeds on this motherboard. The result: you can actually work right next to the machine.

#InterTech #Supermicro #AMD #EPYC #illumos #OmniOS

@jaap Nice beast.. Id love more pictures of the chassis 😸
Inter-Tech 4U-4408 server case

I’ve got yet another Inter-Tech rack mountable server case. I really wanted a few hot swappable disk bays in the front, and buying two removable 4-drive backplanes were more expensive than getting a new case 😮 So I sold the 4U-4129-N and moved Alpha into this case instead 🙂

Cavelab blog

@zolaris @joany That review has a whole bunch of pictures, good reference. My two instances of this case don't have the same problems (as much) as mentioned there. I need to take the rubbers off the Noctua fans to fit them, but then it's alright. The screws/holes need just a little bit of punishment but it works out alright.
There might be a sharp edge here or there but the quality is good enough for the price, definitely not the type of chassis that will murder you or force at least one blood sacrifice like I've seen in the past.

The official site has some pictures of an empty chassis too: https://www.inter-tech.de/productdetails-142/4U-4408_EN.html
They sell more versions with more bays if needed, but I can do what I need at home with 8 bays.

The chassis is racked now so I can't take more pictures, but I did grab these two a while back.

@zolaris @joany Inter-Tech sells an NVMe-compatible backplane for this chassis too. I don't have a requirement for it but it's cool that they have it.

@jaap @zolaris I really like your setup..

I will definitely check out that chassi

@jaap Nice build, do you have any cost figures? A ballpark for the whole setup?

@zolaris Sure, but it's difficult to be exact about it since I usually have some parts lying around. Rome generation EPYC's with Supermicro motherboards go for good prices on eBay all the time (from China). One thing to keep in mind is some eBay EPYC's are vendor locked, but if you grab a combo it'll be alright. This particular set was ~450 EUR.

A new proper ATX PSU will be around ~100 EUR, maybe up to 50% more if you want nice things.

The Inter-Tech chassis can be had for around ~200 EUR if you're in the EU, sometimes cheaper. I had to buy some cables for the storage connections, it depends on your situation. You might have to go from SFF-8087 on an old LSI card for example to SFF-8643 that the backplane uses.

Storage and memory are too use case-specific to say much about it. If the board had two M.2 slots I would have gone for an NVMe mirror for the OS/container/VM storage. In this case I don't care too much because I have a second server 😁

I had some memory still in the stash. Memory prices fluctuate all the time. Usually I'll work with motherboard/CPU combo's that take registered RAM (RDIMM) and I'll try to find those on eBay at the highest speed my platform can support.
I actually looked at building something with a 16-core EPYC 4005, even new those CPU's are really good value. The 16-core version is available with low and higher TDP, pretty interesting for the home because you could build a powerful machine with low power consumption.
There's a nice motherboard from Supermicro that's not cheap but not too bad either:
https://www.servethehome.com/supermicro-h13sae-mf-amd-epyc-4000-motherboard-review/

It's a workstation motherboard but has separate IPMI/BMC, PCIe 5.0, and 2x M.2, good enough for me. However, the price of any decent amount of DDR5 fucks me up. Since I don't really need that recent gear, I went this way.

Supermicro H13SAE-MF AMD EPYC 4000 Motherboard Review

In our Supermicro H13SAE-MF review, we see how this AMD EPYC 4000 series motherboard performs, covering both server and workstation use cases

ServeTheHome
@jaap Thanks for the detailed explanation, i greatly appreciate it. I am wondering have you ever considered to use SmartOS? It could free up the OS drive.

@zolaris I'm not ruling out any SmartOS use in the future. I don't know much about OmniOS and SmartOS yet but I think SmartOS might support some advanced management tools that would be cool to have, related to Triton.

However, the model doesn't make as much sense to me for a small scale. In theory I could have a redundant boot drive with OmniOS and I would have done it if the motherboard supported a second M.2 drive. That feels more robust than a loose throwaway piece of flash storage. I also don't like the idea as much for a shared colocation rack for example. If your hardware doesn't support an internal CF/SD card and you have to use an external USB stick somebody might accidentally or even purposefully remove it.
The main OS isn't really in the way on a 2 TB drive or mirrored drive, and ZFS boot environments seem robust enough.

The SmartOS model I would like more if I had at least 10's of nodes to run with workloads to match that distributed character. My home servers are too much on the pet-side of the pet-cattle scale. I wouldn't shoot one as easily.

@jaap To get rid of the fan warnings simply set the upper and lower fan thresholds with ipmitool.