Agentic use-cases where it has an external test to run get around this 'crap data' problem by testing against an actual source of truth. In effect, they guess, test, refine, guess again.

So, write tests for the behavior you want and then let it sort out how to make it work with the tests. Works great for writing code. Good luck doing this with anything outside of hard science/computer space......

@noodlejetski @codingcoyote I'm well aware of that study. I only know what the actual impacts have been in my experience.

All I can share is the real-world experience I have.
ex: Creating production-quality services implementing business logic between two APIs used to take a few days with learning the APIs, writing the tests, setting up dev and e2e testing environments, integrating with CI, etc. Now, AI does 90% of the drudgery while we're doing meaningful things elsewhere.

@codinghorror I'm right there with you. The increased productivity is staggering when you know how to write the prompts.
1. Pretend you're writing a legal document or contract - say the things that seem obvious and be painfully precise.
2. use the LLM to eliminate tedious tasks entirely.
3. treat it like a smart junior team member you're collaborating with - give it the shape of what you expect the result to be.

Using these rules, what used to take 3 days can be accomplished in 3 hours.

@nixCraft yeah, not at all surprised. You read their methodology and all it takes is 1 page they hit on an entire domain using php for them to count that site. Oh, and they lump subdomains together so 1 legacy wordpress site sitting on a subdomain and the entire site is considered to use php.
Finally, PHP defaults to advertising that it's being used while languages like python don't generally advertise to the world 'this page generated with python'.

So yeah, those stats are useless.