LRwhat if ram buyout is part of a greater attack on personal computing
any other indicators?
Andre Weissflog@lritter IMHO this 'war on personal computing' had been going on in the background for decades with varying intensity, similar to how DDOS attacks happen all the time without bringing the whole internet down (mostly at least). Personal computing might go back to be the niche/hobbyist activity it was in the 70s to 90s, but it won't entirely disappear. I guess it might become an expensive hobby again though :/
@floooh it begs the question what we wrote all this infrastructure for. certainly not so a bunch of locusts can run their datacenters on it, and nothing else.
@lritter hmm yeah... AFAIK the home computer revolution was mostly feeding on an overcapacity of slightly outdated 8-bit chips which suddenly became very cheap at the start of the 80s, and hobbyists and enthusiasts started to build all sorts of awesome things with this 'junk'.
Maybe we'll see a similar Cambrian Explosion after the AI bubble pops and suddenly there's a shitton of just slightly outdated GPUs and memory flooding the market waiting to be used for actually interesting stuff :)
Inga stands with 🇺🇦 🇵🇸
Bornach@IngaLovinde @floooh @lritter
If AI investor funding to build and run new data centres dries up, the cloud compute power will have to be sold off cheaply. Every hacker will be making services that run in the cloud. The fediverse might benefit from this.
Unviable data centres will have to be powered down and parts auctioned off. This will be an opportunity for those resourceful few who can figure out how to repurpose server rack hardware for home use
https://hackaday.com/2022/01/24/domesticating-old-server-hardware-in-the-age-of-shortages/
@bornach @floooh @lritter your link is for general-purpose servers.
The "AI" servers are an entirely different thing from what I understand. Sure, you can salvage a motherboard and CPU and RAM sticks from them, but that's not a lot (like, the entire "AI" industry will only give you several hundreds thousands of relatively weak PCs, if you discard the GPUs).
Most of their hardware, most of their computing power, is in these overpowered GPUs, which are completely useless outside of "AI" loads in hyperscale data centers. Even if you get such a GPU for free, it will still not be worth it even to pay for electricity it consumes while running, not with kinds of the loads you might be able to run on it. There is no "repurposing" these GPUs afaik.
dmi 💽 @IngaLovinde @bornach @floooh @lritter fwiw i don’t think anyone with knowledge has gotten those GPUs into their hands IRL - wouldn’t be surprised if a smart hacker will find a way to make them useful in some weird way :)
also also: the cost will be drastically reduced if you don’t 24/7 peg them, just like with common computers
@IngaLovinde @bornach @floooh @lritter not my point! I doubt the beefy AI GPUs are completely torn off, with nothing in common with a regular unit. hence they may still be acceptable for video encoding, among other things
i choose to not speculate before we have hard info about the internals
@domi @bornach @floooh @lritter but even if they have things useful for video encoding, I doubt they're more powerful than regular mid-range consumer GPUs in that regard? We do know that their main/only selling point is that they have a lot of FP4 performance.
Maybe, just maybe, the enthusiasts will then be able to reflash them with a new firmware disabling all the FP4 things and turning these extremely expensive, powerful, large, and hot GPUs into the equivalent of regular mid-range consumer GPUs, unit for unit. But if market were suddenly to get a million of even free devices that can, with effort, be turned into idk RTX 5050, that's definitely not going to cause any kind of "Cambrian explosion". Gamers already have 100x that number of similar consumer GPUs in their gaming PCs.
My point is: when measured in units, there is not _that_ much "AI" hardware out there. And when measured in computing power, there is a lot, but 99% of it is useless.
LisPi@IngaLovinde @floooh @lritter It is possible to make custom drivers that would expose the memory as swap block devices and to downclock or shutdown as much of the other features of the GPU as possible.
In all likelihood that would still end-up as inordinately power-hungry memory though.
In all likelihood that would still end-up as inordinately power-hungry memory though.
@IngaLovinde @floooh @lritter It would make for a good memory expansion card in desktops that have considerably fewer memory slots and/or no compatibility with other ECC memory.
For servers it indeed doesn't look amazing compared to the standard.
For servers it indeed doesn't look amazing compared to the standard.
@lispi314 @floooh @lritter yeah I was saying this in the context of that "Cambrian explosion" post.
Like, if tens of thousands of enthusiasts having access to 144GB RAM could cause any Cambrian explosion, such an explosion would already be here, because enthusiasts already have access to more than tens of thousands of used servers with more than 144GB of RAM.
Like, if tens of thousands of enthusiasts having access to 144GB RAM could cause any Cambrian explosion, such an explosion would already be here, because enthusiasts already have access to more than tens of thousands of used servers with more than 144GB of RAM.
@IngaLovinde @floooh @lritter I think it'll depend on just how bad the bubble eventually pops and how low the prices go (those servers still aren't exactly available at pocket-money kind of prices).
But you have a point.
But you have a point.
@IngaLovinde @floooh @lritter > It's not like Nvidia is making these GPUs in the billions.
I was kind of expecting a large production, given the whole "filling up datacenters" thing.
I was kind of expecting a large production, given the whole "filling up datacenters" thing.
@lispi314 @floooh @lritter it's large in terms of money, not in terms of the numbers of these GPUs.
Apparently the latest generation costs six digit amount per GPU; selling just a million of them would already make Nvidia one of the largest companies in the world by revenue and not just by valuation.
@lispi314 @floooh @lritter
> those servers still aren't exactly available at pocket-money kind of prices
But they literally are?
Just a random example from ebay: https://www.ebay.com/itm/226945580797 ("HP ProLiant DL360p Gen8 Xeon E5-2630 v2 2.60GHz 192GB RAM 8x 2.5 SFF P420i iLo4" for 180€); another example from another seller: https://www.ebay.com/itm/376838993431 ("Dell PowerEdge R610 Rack Server 2x Xeon X5560 4-Core 2.8GHz 192GB RAM IDRAC" for 150€).
Or https://www.ebay.com/itm/297919718270 ("HP ProLiant DL380p G8 GEN8 Server Motherboard 662530-001 622217-001 + 384GB RAM" for 300€).
HP ProLiant DL360p Gen8 Xeon E5-2630 v2 2.60GHz 192GB RAM 8x 2,5 SFF P420i iLo4 | eBay
Technical data / Technische Daten HP ProLiant DL360p Gen8. Slots for drives / Einbauplätze für Laufwerke Frontseitig: 8 x 2.5 SFF. Das können kleinere Kratzer am Gehäuse sein. RAID support yes / ja. 1 x HP Ethernet 1Gb 4-port 331FLR Adapter.
@lispi314 @floooh @lritter okay, I can agree that if Nvidia actually makes hundreds of millions of these GPUs priced at $200k each (for multiple tens of trillions in revenue), and then after the bubble bursts all these hundreds of millions of GPUs will be sold on ebay for 0.01% of their original price, $20 a piece (or given away for free which is basically the same in that perspective), then yes, a lot of people are going to get ready access to a lot of RAM.
I don't think any of this is realistically going to happen though.
@floooh @lritter
I think there will first be a race to the bottom along the enshittification curve. AI companies cannot convince enough free tier users to start paying so they will have to embed ads and sell promoted answers. Businesses will pay OpenAI/Google to train their chatbot to give a certain answer for specific topic areas, like when a user asks "where should we go out for a meal tonite?", "what is the best medication for a headache?", "what are the reviewer's saying about that movie?"
I think there will first be a race to the bottom along the enshittification curve. AI companies cannot convince enough free tier users to start paying so they will have to embed ads and sell promoted answers. Businesses will pay OpenAI/Google to train their chatbot to give a certain answer for specific topic areas, like when a user asks "where should we go out for a meal tonite?", "what is the best medication for a headache?", "what are the reviewer's saying about that movie?"