Mastodawn

Cursed Homelab Server Upgrade - Part 5

Everything is wrong and it's all broken. *

I can't get an M.2 NVMe drive to even appear on the bus with the adapters I've purchased.

No errors, no warnings, not even a hint that the drive even could be there. iLO says nothing, BIOS says nothing, I've tweaked settings, factory reset the BIOS and tried every tool I have access to.

I've tried all 8 slots, tried it with and without a caddy.

No dice.

The only variable I haven't explored is brand and type of drive. Both drives I've tried are ones that aren't in an active computer right now, which means 2 different Seagate FireCuda 256GB drives.

Options for moving forward right now are:
1. Continue to try to debug this
2. Acquire a different M.2 to U.2 (SFF-8639) adapter
3. Try to acquire a traditional PCI-e M.2 adapter that plugs into a normal slot and have the boot drives internal.

My understanding is that this is all PCIe: the riser card is a passive adapter that plugs into a "32x" slot on the main board and has 4x SlimSAS (SFF-8654) connectors, these go over cables to identical connectors on the backplane, which splits each one up into 2x U.2 connectors for a total of 8. Each U.2 connector has 4 PCIe lanes. The adapter board then adapts that into a M.2 M-key slot which the NVMe drive plugs into. As far as I can tell, the 4 PCIe lanes run straight from the CPU to the drive with basically nothing nothing between other than stuff to keep the signal working. I swear these are nearly metre long PCIe lanes with 6 connectors including the CPU socket.

Ways this can go wrong are:
- somewhere there's a bad connection - I've reseated everything except the CPU so this is very unlikely unless the wires or sockets are bad
- the adapter swaps around PCIe lanes in a way that causes it to not work - I understand that PCIe doesn't actually care, but I could be wrong
- the adapter isn't doing something the backplane needs to see to power up the slot
- the adapter or drive is incompatible - my understanding is that any drive will work

The last two are my best guess right now. A blue LED on the adapter flashes when I plug it in, so either the adapter and drive are drawing too much power and being shut off (most likely the drive) or there is something the slot is expecting that the adapter isn't providing.

* Everything is not all wrong and it's not all broken.

On one hand, I'm frustrated and angry and want to throw the server into a bin and have done. On the other hand, just writing this has helped me immensely in figuring out a next step.

Step 1: try a different drive, I'll be able to pull the drive from our TV computer later tonight.
Step 2: check the pinout of the adapters
Step 3: figure out what the power-looking circuitry on the adapter does

#homelab #cursedhomelab

Show thread

JulianCalaby

Let's take a look at one of those adapters:

Step 1: what the heck is going on with the power supply rails?

TL;DR: 12V from the U.2 connector goes into a step-down converter, which converts it to 5V, which is fed into a second step-down converter to convert it to 3.3V. The 5V input on the U.2 connector is connected to the common 5V rail via what appears to be a diode.

I'm guessing this is done this way so it can be powered by either 12V or 5V and potentially draw current from both. Looking at a similar adapter from a different brand, they seem to be doing only one stage of power conversion, so maybe this is extra fanciness or paranoia or something. It's probably not a big deal.

All grounds on the SATA side are connected together, so that seems legit.

Step 2: what about all the other signals in the power part of the connector?

On your standard SATA power connector, the only fun pin was 11 which could be used to delay spinup of a drive. By the time this evolves into U.2, we have sleep/wake, activity, and two presence-ish lines and a bunch of signals on the "key". The sleep/wake lines go through to the M-key connector, all but one line on the key go somewhere, but both presence detection lines are floating.

I'm assuming at this point that the PCIe lines are hooked up adequately: all the PCIe-related signals on this side of the connector go somewhere on the M key socket.

So maybe it's the presence lines?

Next step: find some documentation on what they're supposed to be/do.

(Obviously what I should be doing right now is dropping $100 on two slightly higher quality adapters from a local retailer, but I'm stubborn.)

#homelab #cursedhomelab

Show thread

JulianCalaby Apr 8

Found a copy of the specification for U.2 and confirmed, my adapter has both presence pins floating which means no drive present. No wonder the backplane isn't lighting up the port!