Jerry 🦙💝🦙Feb 27
As someone with experience in industrial power systems, datacenter design, and a little bit of computing, I find the discussions around CoreWeave’s (bad) earnings outlook fascinating. All the discussion measures compute in terms of megawatts and gigawatts rather than a measure of compute. In 5 years, when the tech underpinning AI is 8x or 16x faster than today (per watt), the value of today’s 800Mw worth of computer will be worth substantially less than 800Mw of compute in 5 years using the technology of that day. Or is this a tacit recognition that the operations per watt are not really changing and the growth is more about power density (how many watts can fit into a 2U server is really what’s increasing). I don’t feel like it’s the latter. Maybe it isn’t supposed to make sense? 🤷‍♂️
Show threadC.Feb 27

@jerry

Whether the first is happening and to what degree I won't argue.

However, the second point is definitely already happening. A "standard" datacenter rack was, not that long ago, limited to about 50 kW. The rise of GPUs in the datacenter led to that growing to 100 kW or more. The rise of LLMs and the corresponding increase of GPUs, and the drastically increasing power requirements of individual GPUs, means that "AI"-intended products like Nvidia's racks are up to 250 kW in some configurations. And they're talking about 1 MW racks in the near future.

That's downright scary. If some tech is working in the aisle and cooling fails, is she gonna get flash-fried before that rack can shut itself down?

Show threadJerry 🦙💝🦙
@cazabon unless they go bonkers with cabling or buss bars, a 1MW rack is gonna need to be fed with something like 50Kv and you’re right- that is basically a flash bomb that makes crappy recipes
Feb 27 at 3:10amWeb
Show threadC.Feb 27

@jerry

I believe they are talking about big busbars, and high voltage to the individual chassis, with DC-DC converters to take it down to the ELV the GPUs use [edit: fix thinko].

800 VDC, I think I read? Maybe they're trying to benefit from the efficiencies of scale from electric vehicle components. I remember thinking "That's still over 1 kA", and that's a unit only a specialized niche of electrical engineers have much practical experience with.

Show threadJerry 🦙💝🦙Feb 27
@cazabon datacenter workers are going to need high voltage PPE 😂
Show threadC.Feb 27

@jerry

I don't envy them that! And nonferrous zippers and button snaps, come to think of it...

#Maxwell #JamesClerkMaxwell