Jason GormanWe're 2.5 years into this gold rush, and I still haven't seen any gold. I've seen people selling picks & shovels. I've seen "gold experts" selling maps to the gold. I've seen CEOs announce they're going "gold-first". I've seen people selling land where they claim there's gold. But no actual gold.
Matteo Capucci@jasongorman uhm assuming this is about AI, what qualifies as gold for you? surely AI is hyped beyond disfigurement but my experience (esp outside academic circles) is that so many people use LLMs daily for their tasks... so there's gold everywhere by that standard.
@mc At massive losses and a huge environmental cost. Who is actually making money from them?
@mc Take Microsoft Copilot as an example. Almost everyone I know doesn't use it and doesn't want it.
Erwin Rossen 🔸@jasongorman @mc Then you know only a very selective group of people. Almost all programmers I know use it, or another AI tool, and they're all happy with it. That it doesn't work for you doesn't mean it doesn't work for everybody.
Mark LevisonBe careful writing bugs faster just increases the supply of problems elsewhere in the system. Just because many people do something doesn’t make it a good idea.
Writing bugs faster isn’t super power
@mlevison @jasongorman @mc I completely agree, but you can say the same about a good IDE: it lets you write code faster, but if you write bad code, that's a bad thing.
That's why I always review the LLM's code manually, and I generate tests for all pieces of code. Of course you shouldn't just accept everything the LLM suggests, but if you use it properly, it is a very useful tool.
Just because many people complain doesn't make it a bad idea.
Strangely I'm just about to unconference this very topic: https://agilealliance.social/@mlevison/114459746587505566
Critical thinking and LLM generated code is harder than we expect.
Consider a few things that might help:
- Start with BBD/TDD style Test First
- Work in small increments that you can understand realistically I can comprehend 20-30 lines of code in a few minutes
- Assume there will an increase in duplication and complexity so refactoring is way more important
....
Mark Levison (@[email protected])
Attached: 2 images AI Magic Beans Friend or Foe Global Scrum Gathering Munich. Unconference offering today and today only in the lobby. Offered at 10:30am and again at 1:00pm. Discuss Three Principles and Four Guidelines for using GenAI and GenAI infused tools. Background. People are writing Code and User Stories with GenAI (strictly speaking, LLM tools). Are they helping or harming their teams? #gsgmun25 #agile #munich #networking
FWIW on the last point there is strong evidence (See GitClear study), that GenAI is increasing duplication and reducing refactoring.
Also from my experience, when asked to generates examples, test cases it always misses key scenarios.
@mlevison @erwinrossen If you also consider the 2023/24 DORA data, it seems that there's an overall net negative benefit in terms of the complete delivery process. LLMs tend to create larger change sets, which exacerbate downstream bottlenecks like longer code reviews, bugs slipping through and merge conflicts.
This is the standard Big Tech model of "disruption" at work. Take something and make it worse, displace the original workforce so there's no easy way back, and lose $billions doing it.
Curiously I said something similar here: https://agilepainrelief.com/blog/is-ai-making-your-organization-fragile-or-more-resilient/
@mlevison @erwinrossen LLMs are the Deliveroo of code generation.