Jason GormanWe're 2.5 years into this gold rush, and I still haven't seen any gold. I've seen people selling picks & shovels. I've seen "gold experts" selling maps to the gold. I've seen CEOs announce they're going "gold-first". I've seen people selling land where they claim there's gold. But no actual gold.
Matteo Capucci@jasongorman uhm assuming this is about AI, what qualifies as gold for you? surely AI is hyped beyond disfigurement but my experience (esp outside academic circles) is that so many people use LLMs daily for their tasks... so there's gold everywhere by that standard.
@mc At massive losses and a huge environmental cost. Who is actually making money from them?
@mc Take Microsoft Copilot as an example. Almost everyone I know doesn't use it and doesn't want it.
Erwin Rossen 🔸@jasongorman @mc Then you know only a very selective group of people. Almost all programmers I know use it, or another AI tool, and they're all happy with it. That it doesn't work for you doesn't mean it doesn't work for everybody.
Mark LevisonBe careful writing bugs faster just increases the supply of problems elsewhere in the system. Just because many people do something doesn’t make it a good idea.
Writing bugs faster isn’t super power
@mlevison @jasongorman @mc I completely agree, but you can say the same about a good IDE: it lets you write code faster, but if you write bad code, that's a bad thing.
That's why I always review the LLM's code manually, and I generate tests for all pieces of code. Of course you shouldn't just accept everything the LLM suggests, but if you use it properly, it is a very useful tool.
Just because many people complain doesn't make it a bad idea.
Strangely I'm just about to unconference this very topic: https://agilealliance.social/@mlevison/114459746587505566
Critical thinking and LLM generated code is harder than we expect.
Consider a few things that might help:
- Start with BBD/TDD style Test First
- Work in small increments that you can understand realistically I can comprehend 20-30 lines of code in a few minutes
- Assume there will an increase in duplication and complexity so refactoring is way more important
....
Mark Levison (@[email protected])
Attached: 2 images AI Magic Beans Friend or Foe Global Scrum Gathering Munich. Unconference offering today and today only in the lobby. Offered at 10:30am and again at 1:00pm. Discuss Three Principles and Four Guidelines for using GenAI and GenAI infused tools. Background. People are writing Code and User Stories with GenAI (strictly speaking, LLM tools). Are they helping or harming their teams? #gsgmun25 #agile #munich #networking
FWIW on the last point there is strong evidence (See GitClear study), that GenAI is increasing duplication and reducing refactoring.
Also from my experience, when asked to generates examples, test cases it always misses key scenarios.
@mlevison @erwinrossen And if they involve calculations, triple-check them.
For the foreseeable future just assume these tools get arithmetic and mathematics wrong.
They might offer the right steps in a calculation (useful). And still get the wrong answer.
Key thing to remember - an LLM is a text prediction tool, not a calculator.
@mlevison @erwinrossen Apparently, they get the answer and *then* they predict the steps that produced it.
We agree on much, but I don''t think that statement is correct.
These tools just predict which token is the next best response to your input.
@mlevison @erwinrossen Indeed. And that include predicting what "chain of thought" produced an answer, apparently.