@asie
The first thing I did of course is try to find if it had copied something - there were not a lot of examples of ASM mandlebrots to go through on GitHub - many were 16-bit, most that were 32-bit used the FPU, or instructions not available on the 386, or some other disqualifier from direct plagiarism.
After coming up empty on Github I spent a fair bit of time pulling down mandelbrot demos from pouet, as they sometimes include source code.
there were clear and apparent differences in every example I looked at - i learned a rather interesting trick for getting a pointer to the VGA framebuffer going through those!
in any case, it was clear that demo-coders were more skilled, keeping everything in registers in the main iteration loop, whereas as the GPT example was using several temporary variables in RAM.
But I was just impressed that it worked at all.
The entire point was a request that it would have failed miserably a year prior , and something that leaned on the side of having the least training data available as possible - but when these companies have scraped every single corner of the internet by now, it might be difficult to pinpoint any particular task that doesn't have some sort of preceding example it can leverage.
It's difficult for me to measure improvement in quantifiable terms other than giving it these sort of challenges - you can see the various scores on things like ARC-AGI trending upwards with every new model, but that sort of thing is a rather abstract measure - what does that relate to in practical terms?
I feel like the AI companies must thank their lucky stars that coding ended up being AI's "killer app". OpenAI would never succeed with something as vapid as Sora as their flagship product.
The greater acceptance of generative AI by programmers is a very interesting phenomenon. There's probably quite a few psychology thesis papers to mine out of that topic. I'm not really ready to be completely cynical regarding the motivations of programmers vs visual artists or musicians. There may be something more fundamental at play.