"TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead."

https://www.latent.space/p/gpu-bubble

#llms #h100 #nvidia #cloud #genai #gpus #cloudproviders

$2 H100s: How the GPU Bubble Burst

H100s used to be $8/hr if you could get them. Now there's 7 different resale markets selling them under $2. What happened?

Latent Space