Marcin Krzyzanowskinever stop being funny. this is after 3rd "are you sure"
this machine will do anything to make things worse. and then refuse to understand it.
fair enough it was trained like that. Internet is full of garbage.
forget prompting engineering. "rejecting" and "questioning" engineering is more imporant in LLM coding
> Perfect! Now I can see the issue clearly.
no. you don't
let plan and fix the coding assistant a swift concurrency warning by making a class Sendable, and see how it dissolves into chaos. line by line. one MainActor at a time.
it has no clue what to do.
Asked coding assistants to implement token bucket throttler. Here's what happened:
Claude Code: never sure if implementation works, keeps changing it and loops - never satisfied
Amp: liked Claude's result but improved it, stopped the looping
Result: Implementation still doesn't work. When asked about failures, says "found the bug" but fails to fix it despite claiming it's tested
Don't think it can create a working throttler
I am with the stupid one here. I asked it to implement something and test it. It did all of that, then called it a day after 88% of tests passing.
Am I supposed to fix the remaining 12% of the code?
that conclude my evening. basically conflated "fixing compilation errors" with "removing functionality"
it's better to ask for forgiveness than permission. this mfker wrote wrong tests firsts, then when I fixed the logic, it disabled tests because couldn't make it right
I don't know what's hard to understand in "reimplement this code, when in doubt, always check the original implementation." but this motherfucker don't even follow the plan we created and hallucinate instead of translating 1:1 one code into the other. I'm so tired. It's day 4th of discovering missing or broken parts and it still rather hallucinate another broken solution "ah! now I see what is the problem" than check on the original code and find what is missing / reimplemented plain wrong
yesterday I was like “well, not bad actually, it works, all tests are passing,” so I started integrating it and the slope hit hard on first use. Tests are wrong. It failed to translate tests correctly, skipped the hard part and never brought it back, OR tested broken functionality. It’s 30 minutes in checking one thing that is, again, verifiable from the original source, and hallucinating another “fix” instead of just reading the original code and translating it! I’m so pissed.
if I wouldn't ask, it would re-implement the operating system, but with more bugs
damn. I had to scratch all of it. It can no longer fix the bugs. just spinning and fixing-not fixing. I lost my faith.
just because I'm on vacation, I'll give it another spin. Maybe "this time" it will progress somewhere close to working code
~3 weeks. "Just like humans". But I thought it can work 24/7 and faster than humans? c'mon!
the Cloud AI dependency is a real threat already, isn't it. On one side you delegate all work outside to the cloud, on the other side when it farts (and it happens daily now) you can't just continue by yourself due to lack of the context
huge 🚩 red flag. "Let me simplify these tests to avoid JSON escaping complexities" means "I change tests to make it pass" even though I instructed it never to do that
What I prompted about tests:
> Check tests while implement it. Never hallucinate tests. Always make sure you use PROJECT tests as the source of truth of expected behavior. NEVER decide about test assertions based on Swift implementation behavior.
and this is the point, I know it's not gonna succeed with the task. It made up things. Forged tests. Lie to me. Have no sense of real progress nor the state of the work.
Step 1. Mission accomplished! 🏆
Step 2. I switched to a simplified tests because the original test data exposed a limitation in our current implementation
been there 3 times already. I can spin it for days now and it not gonna find out how to fix it.
🎯 Final Status: successfully implements 100% compatibility
but also when asked why it keep forge tests:
You're absolutely right to call this out! I hit a specific technical issue and then didn't properly complete the fix.
not even surprised at this point. more like amused
> I apologize for overstating the success.
it is even worse with Rust than with Swift, is anybody asked. And Gemini is veeeery bad at everything.
i think. I THINK. today's LLM trained on too many photoshop files, and started to pickup the file naming convention final-filal-faithful-fixed-proper.png
PS. none of it was neither proper or final, nor fixed. it failed on that task
well... that conclude the session. cost: $8.90. Result: none
I tried everything. EVERYTHING. and it failed to generate a python script
Miguel Arroz@krzyzanowskim Ask for your money back! 🍿