Ben EverardJul 12, 2024
Terence Eden

🆕 blog! “LLMs are good for coding because your documentation is shit”

That's it. That's the post. Fine! I'll expand a little more. Large Language Models are a type of Artificial Intelligence. They can read text, parse it, process it using the known rules of English, and then regurgitate parts of it on demand. This means they can read and parse a questio…

👀 Read more: https://shkspr.mobi/blog/2024/07/llms-are-good-for-coding-because-your-documentation-is-shit/
⸝
#AI #LLM #programming

LLMs are good for coding because your documentation is shit

That's it. That's the post. Fine! I'll expand a little more. Large Language Models are a type of Artificial Intelligence. They can read text, parse it, process it using the known rules of English, and then regurgitate parts of it on demand. This means they can read and parse a question like "In Python, how do I add two numbers together?" and then read and parse the Python documentation. It…

Terence Eden’s Blog
Jul 12, 2024 at 11:35amShare on Mastodon
Show threadAlys BrooksJul 12, 2024
@Edent As interesting as some of the applications AI are, that is what frustrates me. Many of the programming related tasks they excel at, we already knew how to fix: reducing boilerplate needed, providing better documentation, and creating more responsive tools.
Show threadTerence EdenJul 12, 2024
@alysbrooks but, in 40 years of generally available computing, we've not managed to do any of those things.
Show thread⁂ Fish Id WardrobeJul 12, 2024
@Edent Since technical documentation is often wrong anyway, this might at the very least be valid as satire?
Show threadAshok Argent–KatwalaJul 13, 2024
@Edent I’m highly doubtful that LLMs in general process the input using the known rules of English.
But their training data typically have lots of examples of text in English, so they have a statistical association with it, after a fashion.
Show threadTerence EdenJul 13, 2024
@ashok
What's the difference?
Show threadAshok Argent–KatwalaJul 13, 2024

@Edent I think it’s principally one of structure, to me.

One is implicit patterns in a big bag of data; the other would be some deliberate encoding of grammar and other rules.

Show threadTerence EdenJul 13, 2024

@ashok What's the practical difference?

In a specific domain - for example Scrabble - competency only requires recall, not knowledge. https://www.npr.org/sections/thetwo-way/2015/07/21/424980378/winner-of-french-scrabble-title-does-not-speak-french

Show threadAshok Argent–KatwalaJul 13, 2024

@Edent I think that’s a good example of the distinction.

I agree that’s recall, not knowledge.

By the same token, I don’t think a typical LLM knows the rules of English, but it can produce probabilistic sentence-like things.

Maybe it just comes down to meaning different things by “know”.

Show threadAshok Argent–KatwalaJul 13, 2024

@Edent I quite like an argument I’ve heard from Emily Bender that we should be careful around that kind of terminology with LLMs. She presented one version of it for a general audience on Radio 4’s Word of Mouth: https://www.bbc.co.uk/programmes/m001l97m

But I do agree that coding documentation is often awful and wrong. I think the best fix for that is using attention and intelligence, not statistical language models.

BBC Radio 4 - Word of Mouth, Chatbots

Professor of computational linguistics Emily M Bender chats all things bot with Michael

BBC