Sue SmithPeople keep talking about llms killing saas but we were quoted a ludicrous amount for a previously free platform during the week and I'm rustling up an alternative that's completely free aside from a few days of my own labour using the magical automation that is ✨bash scripting✨
The fun part is that I gave claude a few shots at building it, it's a low stakes project, I don't really care about the implementation and just need something that works, but it failed miserably because it's a relatively novel app using niche libraries that aren't well documented and will be almost non existent in training data, but it's a super simple system I could teach a beginner coder to build
Thinking about this again and I think when these tools encounter ambiguity, from the perspective of their logic when there's no way to verify correctness from the training data, instead of coming back to the human and asking them to make a judgment they keep cycling or hallucinate an "answer"
The traditional way of resolving this would be for the human to make a choice they take responsibility for, but since these tools are partially accountability sinks they're not designed for that
The real "human in the loop" would be for the assignment of accountability lol, I find it absolutely bananas how many traditionally risk averse organisations have embraced this stuff without meaningful guardrails, but I do believe the legal shit, compliance, risk management etc will be the thing that enforces some regulation of these practices rather than the quality or reliability of the output itself
I wonder at what point the affected skills and understanding become business critical in different settings, because in practice that's when things might change, I joked about this on linkedin but I could envisage a situation similar to what happened with observability and monitoring tools as everything moved to the cloud but for codebase comprehension, when does it become urgent that your teams know how the fuck your software works
When I look at how effective incident management works and how the tooling is designed to enable human understanding and action, I feel like we're headed towards codebase understanding becoming a more central part of that world
Lord Kelvin@sue
I believe you’re spot-on, this whole thread.
A VC firm provided a (larger than us) company the funds to buy us, & now has a controlling interest in the new whole. We’re being forced to use AI to code. I suspect the VC also has AI investment, the way they ignore arguments about protecting our IP.
It’s gut-wrenching at times, the short-sightedness.
But your story jibes 100%.