“Optimization” isn’t a simple problem an llm could solve, (if llms didn’t create more problems than they solve). What’s the #1 thing making software inefficient today? It’s that we don’t have a good system for creating cross-platform apps except for through the web so half of all desktop and mobile apps just load a chrome window. We already have wildly efficient video compression codecs with hardware acceleration. Idk what you’d even set the “AI” to the task of doing

Optimization is the kind of thankless annoying task that it would be great to put a robot on. But it’s also usually as much an art as a science, so an LLM would struggle to do it right, or at all. You could use an LLM to comb through a bunch of data and give suggestions of common bottlenecks or performance spikes, if you had a good way to generate and collect that data at scale.

Unfortunately, LLMs tend to be really bad at this. They spit out beginner programmer that can search stack overflow a lot type code.

In one example I saw, it did some very expensive processing before a check to see if that processing would even be applicable, and this was a vibe coded project intended to be an “accelerator”. To the vibe coders dismay, even when it “worked”, it was noticeably even worse than the thing it was supposed to make faster.

In pursuit of autonomous development, they tend to stop if the thing barely passes the tests at all. After doing the work to give it specific enough tests to let it retry until passing, they spend thousands of dollars of retry after retry and you are lucky to even get one barely working pass in the end. To try to have it iterate for optimization is going to be way more expensive, especially since it is thoughtlessly trying stuff without a theory of why a difference would be more optimized or not.

Not that I know a thing about programming or game dev. But, from my ignorant position, I wonder if it would be possible to use LLMs to improve character dialogues in games. Imagine being able to type what your avatar wants to say, instead of picking a pre written dialogue option. And then the NPCs will reply accordingly.

Of course that poses some unsavory questions regarding voice acting, but I wonder if that would be a good application for it. I wouldn’t want any VA to get screwed with something like this

Anyway this is the part where you can reply and let me know how ridiculous this idea sounds and why

To mirror what others have already said, LLMs are actually really fucking bad at this.

For starters, all of the most common optimization problems are already silently fixed for you by the compiler. Modern compilers are crazy smart, and can optimize suboptimal code without the developer even having to think about it. That means that the remaining optimizations are already the ones that machines are bad at handling, and LLMs just don’t know how to close that gap.

I confess, I’m the kind of dev who lives for a good optimization challenge, and since I’m required to use an LLM at work, I’ve set them loose on a few of these problems myself. The results are not good. More often than not, optimization tends to be a big-picture endeavor where you’re thinking about how things fit together and if the project can be restructured to avoid bottlenecks. AI seems to be the weakest at big-picture stuff, so it tends to hone in on small details that aren’t going to give you much improvement. What they will do, however, is add lots of verbose code in pursuit of chasing a few extra milliseconds here and there, which is sub-optimal in its own way.

Readability and maintainability are both their own separate dimensions of optimization because they speed up the process of debugging and future development. AI does not optimize for these AT ALL. It often tries to compensate for this by filling the code with comments, but this actually makes the problem worse when your codebase becomes cluttered with hundreds of useless annotations like this:

//Initialize the app

InitializeApp();