Sue SmithPeople keep talking about llms killing saas but we were quoted a ludicrous amount for a previously free platform during the week and I'm rustling up an alternative that's completely free aside from a few days of my own labour using the magical automation that is ✨bash scripting✨
The fun part is that I gave claude a few shots at building it, it's a low stakes project, I don't really care about the implementation and just need something that works, but it failed miserably because it's a relatively novel app using niche libraries that aren't well documented and will be almost non existent in training data, but it's a super simple system I could teach a beginner coder to build
Nick@sue Yeah, my own experiences of testing a different LLM have been similar.
Couldn't find docs for how to do something with an obscure library, tried asking LLM for an example of how to do the thing.
It obligingly produced code but completely hallucinated the only part I needed info on. When I challenged it on this, it cheerfully admitted making it up.
A human developer who repeatedly behaved like this wouldn't stay in their job for long...
RootWyrm 🇺🇦
@pythoneer @sue my favorite fucking part is that what is now also happening as a direct result is that people who were barely documenting things to begin with, now just don't document them AT ALL. "Oh, just ask the chatbot!" Nevermind that the chatbot is always fucking wrong. It also doesn't know anything that happened after 2023.