implr4d ago

In light of all the recent hype, I decided to see how LLMs deal with truly weird problems, not just webslop.

I assumed they'd fail in a spectacular and entertaining way at debugging linux audio. Turns out, at great token expense, a modern LLM *can* debug problems involving lots of intricate bullshit with poor documentation.
I'm not sure how to feel about this.

Show threadimplr
context: I got a Thinpad T14gen6 AMD recently and installed #gentoo on it. Besides the typical oopsie of forgetting to compile in a touchpad driver or something, it works almost flawlessly. Almost.
The mute LED on the f1 key doesn't work.
I tried some easy things with no results, so fuck it, let's have the clanker try. It'll be fun to watch it fail.
Apr 5 at 11:26amWeb
Show threadimplr4d ago
I gave Anthropic 20 bucks and ran Claude Code (with manual approval of every shellout) with a pretty basic prompt (this is the problem, the running kernel is in /usr/src/linux/, any other info you can get from sysfs or pactl, figure it out).
It spent hundreds of thousands of tokens "investigating", then ran out of daily quota.
I considered it a failure at this point, but decided to give it another try next morning.
Show threadimplr4d ago

Turns out I was using the stupid model (Sonnet vs Opus) by default, so I switched. Another 150k of context (mostly loading what the previous one did) later, it poked a few more files in sysfs and it figured it out.

I have mixed feelings about this.
Could I have done this myself? Yes.
Would it have taken me an entire day? Absolutely.
Would I learn a lot? Sure.
Is it worth a day of my free time to know the intimate details of ALSA codec-LED binding? Eh, probably not.

Show threadWolf480pl4d ago

@implr

idk if it works that way for you, but I'd add two more questions to the list:

would you be having fun investigating it yourself?
(I'm guessing no)

having spent all day investigating it, would you feel satisfied, or angry at the code having "such an obvious" mistake?

Show threadimplr4d ago
@wolf480pl
1. i got frustrated in my first manual attempt, so probably not. if i got to the reading kernel source part (what the llm did) it'd be interesting but still frustrating because:
2. there wasn't even a code mistake, it's purely user error. i flipped the wrong kconfig flag months ago and didn't know. so i'd be pretty angry at myself