If open models on local hardware were more cost effective and competitive, it would be obvious that this is such a superficial approach. (I mean, it still is obvious but what are ya gunna do?)

We would be doing the same general loop, but fine tuning the model overnight.

I still think the current LLM architecture(s) is a very useful local maximum, but ultimately a dead end for AI.