every time someone tells me about using llms for codegen, all i can think about is that old adage
"debugging software is twice as hard as writing it, so if you write to the best of your ability, you're not qualified to debug it"
sure enough i know several "don't call me a programmer" friends who find it easier to get starter projects off the ground, and these tools have helped a whole bunch
but they're the first to admit "if it doesn't work i don't know how to begin fixing it"
@tef so, I haven't had to debug this yet, because the integration work is not totally trivial, but I think I had my first success with this recently
I asked it to do a thing which github actions yaml that I knew how to do, but would be a truly mind-numbingly tedious amount of typing; about 100 lines of yaml all told
I asked it to do the thing, and it looked like it basically got it right, and it was a 1-line prompt for 100 lines or so of repetitive, basically branch-free yaml
@glyph i get this but this feels like a solution to a problem we created for ourselves, rather than a problem we face
"what do we want" "for loops" "how do we want them" "either recursively templated yaml and/or a fancy markov chain"
@tef hey _I_ didn't make github actions
somebody showed up and said "hey would you like about half a million dollars worth of free computer for open source every year" and the cost was having to generate reams of tedious configuration garbage and honestly it's a pretty good trade, especially if they will give me an extra 100k worth of free computer to compute the tedious configuration garbage too
@glyph god, same
i tell myself it's not m4 at least
i mean, sure, i'd train a neural network to avoid using autotools
but also i'd just avoid using it :-)
@tef relatedly; iirc Microsoft claimed that LLMs was helping their engineers deliver code 6% faster. Two things immediately struck me:
1. That’s a pretty mediocre boost. There are probably way more efficiencies to be gained in just organizational quality, let alone other tooling.
2. I don’t believe them. I suspect they’re sweeping a lot of nuance and tech debt under the rug to achieve a result to justify other actions (investments, valuations, layoffs, etc.)
@tef I wouldn't say it's any worse quality code than hiring in a big consultancy or offshore who spend millions and screw it up anyway, at least the GPT will attempt to write tests or documentation if you ask it.
There's always someone's fuck ups to fix - I do agree LLMs are there to give everyone just enough knowledge to be dangerous without guardrails.
It's a failure of leadership to allow that stuff anywhere near production without testing and validation.
tef
Glyph
Cassandra is only carbon now
Hazel Weakly
wirepair
⛈️ Information ⛈️
sabik
Ashton
Billy Smith
Dan Hulton
Chris Johnson
Tane Piper ⁂