@RichRARobi I know what you mean.

A former colleague of mine was using a neural network to classify bird pics from his bird-feeder cam using a laptop 15 years ago. The NN wouldn't run on the RPi that took the pics, but it was fine on a (then) old laptop.

But that was a very focused NN, trained on data they had produced, and classified themselves. His setup used to take a few seconds for each image.

Fast forward to now, and I have a friend messing round with an RPi4 based "small model" LLM for hazard avoidance on a "toy" robot. He gets a few tokens per second.

But to run the full "enterprise grade" LLMs with "billions" of parameters like ChatGPT et al, you need specialist hardware, and lots of it - especially for the training part, where if you want to finish in less than a lifetime, you need petabytes of data and terabytes of RAM and thousands of GPUs, then you need more hardware to RUN the LLM Neural Networks, plus all the infrastructure to front it - web servers, app servers, live, staging and test systems, crash-proofing server redundancy....

After saying all that, I think the main reason all that infra is "needed" is to make number go up: if it looks like there is a lot of investment going on, gullible investors are going to get gulled into buying shares to make that happen. They couldn't possibly be investing that much cash in it unless they were expecting a decent payoff eventually! Right?