My experience with generative-AI has been that, at its very best, it is subtly wrong in ways that only an expert in the relevant subject would recognise. So I don't worry about us creating super-intelligent AI, I worry about us allowing that expertise to atrophy through laziness and greed. I refuse to use LLMs not because I'm scared of how clever they are, but because I do not wish to become stupider.
I will say one thing for generative AI: since these tools function by remixing/translating existing information, that vibe programming is so popular demonstrates a colossal failure on the part of our industry in not making this stuff easier. If a giant ball of statistics can mostly knock up a working app in minutes, this shows not that gen-AI is insanely clever, but that most of the work in making an app has always been stupid. We have gatekeeped programming behind vast walls of nonsense.
We seem to have largely stopped innovating on trying to lower barriers to programming in favour of creating endless new frameworks and libraries for a vanishingly small number of near-identical languages. It is the mid-2020s and people are wringing their hands over Rust as if it was some inexplicable new thing rather than a C-derivative that incorporates decades old type theory. You know what I consider to be genuinely ground-breaking programming tools? VisiCalc, HyperCard and Scratch.
You know what? HyperCard was a glorious moment in time that I dearly miss: an army of non-experts were bashing together and sharing weird and wonderful stacks that were part 'zine, part adventure game and part database. Instead of laughing at vibe-coders, maybe we should ask ourselves why the current state-of-the-art in beginner-friendly programming tools is a planet-boiling roulette wheel.
@jonathanhogg HyperCard was *amazing* and I don't understand why there's nothing like it anymore. It was like building programs with Lego. Just snap things together, write your program in a very natural language, and do incredible things. It was so easy to double click on something and add a few lines of code. I remember also having fun with the flexibility of the language and constantly trying to see what different syntax I could get away with.

@fozztexx @jonathanhogg there are quite a few, Lego even have a visual programming language for their smart bricks (I think Python is officially supported, unlike not quite C for Mindstorms).

There's also a visual language for a smart RC/drone controller built by that one guy :) iforce2d is the guy.

But they're generally hard to to anything nontrivial with and very hard to debug. Like Excel/Calc... so easy to have subtle errors even in simple programs that it's considered inevitable.