Marcin Krzyzanowskinever stop being funny. this is after 3rd "are you sure"
this machine will do anything to make things worse. and then refuse to understand it.
fair enough it was trained like that. Internet is full of garbage.
forget prompting engineering. "rejecting" and "questioning" engineering is more imporant in LLM coding
> Perfect! Now I can see the issue clearly.
no. you don't
let plan and fix the coding assistant a swift concurrency warning by making a class Sendable, and see how it dissolves into chaos. line by line. one MainActor at a time.
it has no clue what to do.
Asked coding assistants to implement token bucket throttler. Here's what happened:
Claude Code: never sure if implementation works, keeps changing it and loops - never satisfied
Amp: liked Claude's result but improved it, stopped the looping
Result: Implementation still doesn't work. When asked about failures, says "found the bug" but fails to fix it despite claiming it's tested
Don't think it can create a working throttler
hence my question is: how did it pass the leetcode?
from the backstage: the fix indeed did not fix anything, and that was already the 4th statement that this time this is the fix - each time not fixing a different thing.
LOL. First it implemented delegate, then offer alternatives. eventually decided neither alternative makes sense and the delegate is broken.
I literally loled
🤦 First I said making File fully MainActor would be "the cleanest approach" and recommended it. Then when you asked if it's better, I said "not really"
I trust you bro. I trust you with my life
Rob Napier@krzyzanowskim I have at least been impressed with the models' ability to undo the damage they do.
Konstantin 🔭@cocoaphony @krzyzanowskim unless you make changes on the side while the agent is working in the same repo 😂🙈. Otherwise yes, they git their way back to the previous state
@iamkonstantin if there is git, yes. otherwise if compact happen in the meantime, the data is lost or restored faulty
@krzyzanowskim must be a skill issue /s
Martin Du4@krzyzanowskim rewind is the exit door. 🚪
Ivan Cantarino@krzyzanowskim the saga continues 🤣
@krzyzanowskim in a near future, AI will present you this thread in the AI court 😅
I am with the stupid one here. I asked it to implement something and test it. It did all of that, then called it a day after 88% of tests passing.
Am I supposed to fix the remaining 12% of the code?
that conclude my evening. basically conflated "fixing compilation errors" with "removing functionality"
it's better to ask for forgiveness than permission. this mfker wrote wrong tests firsts, then when I fixed the logic, it disabled tests because couldn't make it right
I don't know what's hard to understand in "reimplement this code, when in doubt, always check the original implementation." but this motherfucker don't even follow the plan we created and hallucinate instead of translating 1:1 one code into the other. I'm so tired. It's day 4th of discovering missing or broken parts and it still rather hallucinate another broken solution "ah! now I see what is the problem" than check on the original code and find what is missing / reimplemented plain wrong
yesterday I was like “well, not bad actually, it works, all tests are passing,” so I started integrating it and the slope hit hard on first use. Tests are wrong. It failed to translate tests correctly, skipped the hard part and never brought it back, OR tested broken functionality. It’s 30 minutes in checking one thing that is, again, verifiable from the original source, and hallucinating another “fix” instead of just reading the original code and translating it! I’m so pissed.
if I wouldn't ask, it would re-implement the operating system, but with more bugs
damn. I had to scratch all of it. It can no longer fix the bugs. just spinning and fixing-not fixing. I lost my faith.
just because I'm on vacation, I'll give it another spin. Maybe "this time" it will progress somewhere close to working code
NeoNacho@krzyzanowskim me at standup
Ken@krzyzanowskim I frequently see ChatGPT interested in trying to chase down esoteric JavaScript logic issues (I do a little scripting in Obsidian for knowledge management) rather than, like, focusing on the fundamentals of the program logic.
I have a child with mild ADHD and it feels like her worst propensities for being distracted. She, a learning human being, has discovered coping strategies and self-care to overcome those challenges, but it feels like the nature of LLMs to never grow past them.
Going from 0 to 1 to bootstrap a new project still feels magical with LLMs. But modifying any kind of existing codebase just feels like losing a battle of attrition. 🫠
Joe Groff (1M Context)@krzyzanowskim "coding agents make you 20% slower" actually statistical error. agents marcin, who spent two weeks arguing with a coding LLM, is a statistical outlier and should not have been counted
@joe you're not wrong. obvious "prompting issues" on my side. and I can't write spec. I can't do the proper plan. It would work 100% if I only do all thing right. I'm sold on that idea and bet half of a bitcoin it's sentient
Jeff Watkins@krzyzanowskim @joe I wonder if it would be cheaper and less frustrating to hire a junior developer. At least then you’d be helping another human improve.
@jeffwatkins @joe As an indie the human hiring process is quite a steep comparing to "easy" launching the ai app. As a company I hanve no doubt rhe junior is 100x more valuable asset than LLM agent. no doubt at this point.
Mieszko Ślusarczyk@krzyzanowskim now it’s starting to look like the Beckhams meme
Alex (Podcast Guru)@krzyzanowskim all this AI-driven development looks more and more like a comedy 🤣
@algrid it could make up for a short standup
@krzyzanowskim just like a real programmer 😂
@krzyzanowskim ha! The folks at fly made a thing for Phoenix - you can sign up to get a Claude running in a VM, preconfigured with “best practices” etc but the thing that it controls the entire OS and can sort itself when to sudo or install things “safely” is fun. Like, “sudo all you want but fix the tests” 😂 https://phoenix.new
Matt Barlow@krzyzanowskim I really hope the controls for whether sudo is allowed are implemented in normal code and not part of the LLM itself 😬
Colin Cornaby@krzyzanowskim Sure the Xcode refactor tool is ever so slightly more reliable - but you can't yell at it when it goes wrong.
@colincornaby can't wait Xcode 26 chat!
@krzyzanowskim this is the way
Ellen Shapiro@krzyzanowskim I would have fired it quite a bit further up this thread
Paddy O'Brien@krzyzanowskim a few weeks ago I was doing hack days at work and cursor made edits to some tests I had told it to ignore because it couldn’t make them pass. It fucked up all of the tests then just deleted the file because we proved it already worked
@krzyzanowskim I was just watching this tonight and thought of you, my friend. (I hope we can have coffee again soon.)
'Forbidden' AI Technique - Computerphile
softmaus@krzyzanowskim It’s theraphy. Only thing is, you are the therapist.
@softmaus I have disconnect from the ai slop blogposts and reality
Jason HowlinFor any time I might save, I spend twice that arguing with the AI
@jasonhowlin pretty much. It save me actual typing when I feel lazy
Aleksandar Vacić@jasonhowlin I bail out as soon as I see it’s starting to halucinate nonsense and start over. There are no feelings to be hurt here so 🫢