Tony Arnold

I just used GPT-5.3-Codex-Spark for a simple implementation and it ran out of usage in under 5 minutes.

*This* is what will ultimately cause the current business model to fail outside of big tech and enterprise.

You can't claim the correct usage of an LLM is to pop it in a verification loop, while also charging a per-token access/usage fee that means it's unusable for that purpose.

Perhaps in a few years we'll see local LLMs that work well on consumer hardware with the same capabilities?

Mar 8 at 12:53amIvory for Mac
Show threadTony ArnoldMar 8
For now, the business model of OpenAI and Anthropic is a wonderful way for the rich to gatekeep the means of production.
Show threadHuw RowlandsMar 8
@tonyarnold this is my main problem with the whole thing. I’ve gotta pay a subscription to some MAGA muppet to just participate
Show threadColin CornabyMar 8

@tonyarnold It’s also not clear if they’re making money at API rates. They don’t supply “all in” numbers and training is getting expensive. They’re definetly losing money at the monthly rates. That means they can’t afford to supply the models to anyone.

The increased training costs will be a problem for the open models too.

Show threadTony ArnoldMar 8

@colincornaby I had someone at work straight-faced say that the costs of these tools and workflows are approaching zero. Their claim was that we were getting more for our money over time, but that is a very manipulative definition of “approaching zero”.

It will be very interesting to see what happens here around pricing/access over the coming years.

Show threadColin CornabyMar 8
@tonyarnold What? I don’t think there is any way that is true. The last estimate I saw is a Claude Code users with a $200 subscription, who maximizes usage, costs Anthropic $5000. No way are these things approaching zero. They require multiple 80k GPUs even to serve one session.
Show threadTony ArnoldMar 8

@colincornaby i think it’s quite likely the current crop of companies will collapse, much like the blockchain-focused companies before them did.

We’ll be left with some really interesting tooling regardless of the existence of pay-per-play orgs like Anthropic and OpenAI.

Show threadColin CornabyMar 8

@tonyarnold Yeah, it depends how much companies like Deepseek are actually stealing data from OpenAI and Anthropic. If that is going on they may not be able to train further models either.

I think the whole game is really human engineer replacement. _If_ you can just get rid of the humans and their salaries those rates look more sane.

Show threadTony ArnoldMar 8

@colincornaby I have credible doubts about whether replacement of human engineers is even possible.

Don't let reality get in the way of a good investor pitch, though.

Show threadMΛTTMar 8
@tonyarnold my experience with it is the same. I think your prediction for the future is about right. Make sure you buy machines with a bit more ram in them.
Show threadKarstenMar 8
@tonyarnold I experienced the same with Claude’s extra charge. If you add your monthly fee as extra charge this will not allow you to do any extra work when your limit is reached. It‘ll just burn these extra tokens faster than you’d expect.
Show threadJoe Groff (1M Context)Mar 9
@tonyarnold on the flip side, with more verification guardrails you might be able to get competitive results out of a smaller local model. if you're lucky enough to have a macbook pro or mini with 48gb of ram, that can fit qwen3.5-35b-a3b. its initial results have not impressed me, but if you make it try again until some tests pass you can eventually get something correctish out of it
Show threadTony ArnoldMar 9

@joe I'd love to see local models become the "standard" for using this technology, but I have had similar results on a 64Gb RAM M3 Max MBP — it's too slow to really be useful.

I hear a lot of folks are having good results with that model, though - I just can't replicate them at any kind of speed.

Show threadColin CornabyMar 9

@tonyarnold @joe I was using Qwen 3.5 35b last week to do some OpenGL and Metal. Asking it to do larger tasks that involved shader architecture led to some... interesting results. But it at least put out some shader code that made for a halfway decent starting point once it was cleaned up.

I sort of wish we could go back to "these things are weird little assistants" instead of "these things are complete software engineers."

Show threadTony ArnoldMar 9

@colincornaby @joe saying they're something when they are not does not make them that thing.

I know the US has fallen into a bit of a post-truth era, but these things *are* assistants for any kind of professional use.

Show threadColin CornabyMar 9

@tonyarnold @joe I think the hard part is software development is so diverse.

If you work on login pages or todo apps these things do look a bit more like software developers in a box.

I know web front end got way more complicated over the last few decades. But I still think it was a mistake to pull everything under the software engineering umbrella.

Show threadJason HowlinMar 14
@tonyarnold talk about switching costs! After companies lay off engineers, the AI companies are free to raise prices to reflect actual costs. Companies will have no choice but to pay.