mnl mnl mnl mnl mnlMay 14

You can observe the moving of goalposts that seems to be a constant when dealing with #llm #llms and coding, right now:

We went from “llms can’t find real bugs, just a hallucinated mess” to “llms can find valid bugs but are not able to construct exploitation paths” to “llms can find real security issues and constructs exploitation chains, but they’re not real security bugs exploitable at large” to “some are real and exploitable, but you need xyz to do it” within like, a year.

Predicted trajectory “some are real and exploitable but you need knowledge to properly prompt the llm” to “security is not about finding exploitable bugs” to “why are all the techbros suddenly rushing into software security” to “we are banning all llm based security tooling”.

(PS: just saw another one on the timeline just now: “llms can find legit, exploitable bugs, along with the relevant exploit, but how many false positives did it raise?”)

(PPS: another one: “they can find new attacks, but not new attack techniques”)

Maybe you are not yet used to this cycle, but this hopefully explains why some of the voices much less invested in the hype are genuinely worried, and the mythos announcement is just the opportunity to have a wider discussion. Mythos is hype yes, the problem was already here and won’t go away even if mythos is a “dud”. The scary part is the trajectory.

larsbornMay 14
Show threadmnl mnl mnl mnl mnl

For context:

We went from “cannot understand the difference between C and PHP” to “can sometimes write a valid function” to “works reasonably well to work on single files” to “can build a full greenfield app but needs extensive guidance on architecture and APIs” to “can build a full app with an engineer in the loop and build on top of it for a few weeks” to “decent at architecture and can build smaller systems without guidance” in 3 years.

But when I was trying to talk about labor issues and it being a paradigm shift for the industry at large, the standard response was that I was deluded and spreading FUD. The take that the tools are useless has been constant too, except the goal posts constantly move to whatever the current state of the art. Another take that never dies is that using llm based tools somehow can’t involve skill, that there is no difference between the prompting of an experienced software engineer who has spent years working with llms and the 3 prompts one has put into a random model “to try things out”. Imagine someone coming to like Elixir from Java, typing a few classes in Java, runs it and gets errors and say “elixir is kinda useless, all I got to run was this super barebones program after 17 tries and lots of compile errors”.

Whether one like using these tools or not (especially if you don’t like them), and especially if you are relatively new to them, spend just a few minutes or hours to compare how far you get with llama (the OG) and pure copy paste by hand, to a newer 8B model in an agent harness, to a model like glm5.1 to gpt5.5 or opus4.6 in a harness.

That’s the last 2 years in a bottle.

#llm #llms #genai #claude #vibecoding

May 14 at 12:48pmWeb