Paul CantrellAug 13, 2024

In most cases, LLMs will not replace humans or reduce labor costs as companies hope. They will •increase• labor costs, in the form of tedious clean-up and rebuilding customer trust.

After a brief sugar high in which LLMs rapidly and easily create messes that look like successes, a whole lot of orgs are going to find themselves climbing out of deep holes of their own digging.

Example from @Joshsharp:
https://aus.social/@Joshsharp/112646263257692603

j# (@[email protected])

Yesterday we had another example of LLMs creating support issues for us. User: "hi, how do I do this thing? Your docs say I can go here and change some options, but there's no settings there" Me: "that's right, we don't have such a feature, but also we don't say you can do it in the docs, where did you read that?" User: "oh I didn't actually read the docs, I asked 'AI' and it hallucinated this answer. Sorry!" At this rate I'm looking forward to 2025 when I'll be spending 100% of my time doing support to correct falsehoods about our app made up by LLMs

Aus.Social
Show threadPaul CantrellAug 13, 2024

Those who’ve worked in software will immediate recognize the phenomenon of “messes that look like successes.”

One of my old Paulisms is that the real purpose of a whole lot of software processes is to make large-scale failure look like a string of small successes.

Show threadPaul CantrellAug 13, 2024

The crisp “even an executive can understand it” version of the OP is:

⚠️ AI increases labor costs ⚠️

(“Why?” “Because it’s labor-intensive to clean up its messes.”)

Show threadPaul Cantrell

I said “the purpose of a whole lot of software processes is to make large-scale failure look like a string of small successes.”

Huh? What does that look like??

It looks like this:

✅ Meetings held
✅ Plan signed off
✅ Tests passed
✅ Iterations iterated
✅ Velocity increased
✅ Thing implemented
✅ Checkpoints checked
✅ Thing released
✅ Blinkenlights blink
✅ Line goes up
✅ Thing updated
❌ Software never •really• solves the problem it was supposed to solve in the first place, creates more problems

Aug 13, 2024 at 8:14pmWeb
Show threadPaul CantrellAug 13, 2024

or ❌ Problem the project was supposed to solve in the first place was the wrong problem

or ❌ Nobody actually wanted it

or ❌ We totally failed to understand the real effect of implementing this

or ❌ The goal was designed to benefit some individual / faction within the company, not the mission

or ❌ The goal was designed to benefit the bottom line / investors / some horrid systemic evil, and is net harmful to humanity / the world

(Yes, I consider that last one a failure too.)

Show threadPaul CantrellAug 13, 2024

My most hilarious example of “large-scale failure looks like a string of successes:“

Years ago, I worked on a project for retailer Megacorp Y to sell their house-branded cables on Megacorp Z’s online sales platform. It was an integration project: wire up inventory, wire up payments. The tech side was sloppy (weird, ancient APIs, Z’s official API involved •FTP• transfers (yes, really)), but ultimately quite tractable.

The problem? Internal conflict between ambitious humans.

Show threadPaul CantrellAug 13, 2024

One team at Y did inventory, and a different team did payments. Both teams had ambitious (and pretty jerky) managers who wanted to control the •whole• project so they’d get credit for it when it launched. Both managers therefore wanted the other team’s side to fail.

We’d have meetings between the inventory and the payment teams where the engs would say, “We could do this!“ “Oooh, and then we could do this!” And the managers would suddenly cut in: nope nope nope, we can’t have this just •work•.

Show threadPaul CantrellAug 13, 2024

I cut the cord on that contract — the agreement had only even been for me to architect it and lay the technical foundation — and left figuring it would never, ever see the light of day.

About a year later, somebody came up to me at a conference.

THEM: You’re Paul? You worked on this project at Y??

ME: Yes…

THEM: You won’t believe it: it actually •got released•! Against all odds, it went out the door and into production!

ME: 😮😮

Show threadPaul CantrellAug 13, 2024

THEM: And it worked! Your code was great! In fact, it was so successful that it made several million in its first few weeks!

ME: 😵

THEM: …and it was so much money that it showed up as a line item on a report to the CEO, and the CEO said, “What's this?,” and when they told him, he said, “We're doing WHAT?!? Why are we selling our house-brand products on their platform??? Shut that down NOW!!!” and they pulled the plug on the whole thing!

ME: 🤣🤣

I guess neither manager got that promotion.

Show threadRobin OsborneAug 13, 2024
@inthehands *Had me in the first half, not gonna lie*
Show threadPaul CantrellAug 13, 2024
@rposbo Had •me• in the first half — and I was there!
Show threadCassandrichAug 13, 2024
@inthehands Monoprice cables aren't on Amazon anymore?? 😂
Show threadPaul CantrellAug 13, 2024
@dalias
I can neither confirm nor deny any specific companies involved in this story, but you have exactly the right idea
Show threadAlaric Snell-PymAug 13, 2024
@inthehands hahahah! I was once brought in as a technical consultant by a primarily sales org that, wanting a product in a new sector to sell, had gone out and bought the two biggest competitors in the space and told them to merge to make a really good product. And they didn't understand why these two teams that had been built around beating each other and developing contradicting worldviews to differentiate their products couldn't just merge their products...
Show threadPaul CantrellAug 13, 2024
@kitten_tech
As the Minnesotans say, uff da
Show threadJima Aug 14, 2024
@inthehands All the locals: "Mm-hmm; 'Y,' 'Z.' We hear you."
Show threadPaul CantrellAug 14, 2024
@jima
Your guesses about Y and Z are probably wrong. Unless they aren’t. They might not be. Or not. I can neither confirm nor deny.
Show threadJima Aug 14, 2024
@inthehands
Show threadJenniferplusplusAug 13, 2024
@inthehands Notably, AI is itself software and is subject to the same forces that produced this list. I'll leave it as an exercise to the other readers to figure out which of these red Xs applies in that case.
Show threadCassandrichAug 13, 2024
@jenniferplusplus @inthehands All of them.
Show threadjmeleskyAug 13, 2024
@jenniferplusplus @inthehands Probably easier to ask “which don’t apply?”
Show threadmaswanAug 13, 2024

@inthehands

Don't forget the old Nokia management style:

or ❌ The goal was designed to harm some individual / faction within the company, not benefit the mission

@kissane

Show threadPaul CantrellAug 13, 2024
@maswan @kissane
Ha, coincidence! See the story I just posted downthread
Show threadmaswanAug 13, 2024
@inthehands
Hah, yes indeed.
@kissane
Show threadCassandrichAug 13, 2024
@inthehands Usually all of the above... 🤦
Show threadMikeKAug 13, 2024

@dalias @inthehands

And you have just, very adequately explained why AIs can't write useful code

Show threadGeoffrey Conley 🏳️‍🌈 🖥️ 🚴🏼 ☕Aug 14, 2024

@inthehands okay I'm booking this whole thread, it's so succinct and relatable. Honestly a nice reminder of the scope at play.

I'm curious, how pervasive would you say this is? Is this like every project in every org? Or half of half? Struggling to see past my own experiences, which are honestly a lot of this on repeat.

Show threadPaul CantrellAug 14, 2024

@geoffreyconley
I remember many years ago having an animated discussion with coworkers about what percentage of all corporate software projects are hoaxes (where “hoax” means “never really succeeds, even if it’s presented as a success).

Our conclusion is that we have no idea, and no way of finding out. Too many projects, wildly different places, no way of evaluating the question.

Truly, I have no idea. I just know this: it’s not rare.

Show threadGeoffrey Conley 🏳️‍🌈 🖥️ 🚴🏼 ☕Aug 14, 2024

@inthehands that seems fair enough! Thanks for your response!!

Sorry to ask, but in terms of personal philosophy, do you aim to find projects and employers that AREN'T this way? Do you end up accepting that it happens, and just do your part?

Show threadPaul CantrellAug 14, 2024

@geoffreyconley
The main things I look for in choosing a project:

- Cool people
- Worthy goal
- Fun problems to solve

With 1/3 I can survive. 2/3 is a great project.

I suppose I do consider the odds of the project being truly successful — but mostly I take a kind of Camusesque perspective: it’s necessary for orgs to try things, for projects to crash and burn, for people to explore what turn out to be dead ends. In the end, I just want to have been glad to have done the work.

Show threadPaul CantrellAug 14, 2024

@geoffreyconley
For example, with the downthread story about Megacorp Y, my main regret is •not• about the CEO pulling the plug on everything I did a year after I left. No, that’s just hilarious!

My regret is that I found out later one of the managers had been really emotionally abusive to some of the other devs, especially the one woman on the team, when I wasn’t present. I wish I’d realized that while I was there! I wish I’d fought back! But I was oblivious to it. Big regret.

Show threadRoger MooreAug 18, 2024
@inthehands
My favorite is when the project targets a solved problem. It looks like a success because it achieves the goal as defined, but it was still a big waste of resources. This seems incredibly common to me.
Show threadIrenes (many)Aug 13, 2024
@inthehands notice how techno-solutionism comes very close to acknowledging that this is what's going to happen, but casts it as a good thing (which it is, for profit)
Show threadmisomunjeAug 13, 2024
@inthehands I have worked on several successful software projects and the checklist looks the same with a ✅ for the last item.
Show threadPaul CantrellAug 13, 2024
@misomunje
Yes, successful projects do exist! And in my experience, without exception, they involve a whole team with eyes on some goal that is larger than the process they’re using to get there.
Show threadMarc CrileyAug 13, 2024

@inthehands I've told people from time to time that a dirty secret of corporate software testing organizations is that the product is released when it passes the test cases, which *should* overlap with the desired functionality, but 🤷

(I got to write lots of bug reports when we were turned loose for 'free play' testing)

Show threadAndresAug 13, 2024
@inthehands Wait, are we talking about software or state DOT road/highway projects?
Show threadAndresAug 13, 2024
@inthehands *loads up some traffic modeling software*
Show threadBill SeitzAug 13, 2024
@inthehands success theater
http://webseitz.fluxent.com/wiki/Theater
cf proxy metric
Theater

Theater

WebSeitz
Show threadMy name is GordoAug 14, 2024
@inthehands Severely underrated banger of a post.
Show threadchetmanAug 14, 2024

@inthehands @mhoye Oh, do I ever have Thoughts here.

For the last 17 years I’ve worked in the EVMS software space. Earned Value is tricky to implement but it WORKS. It requires rigor in determining what has actually been accomplished— and just spending time and money isn’t it.

Software projects are horrible at this. They shouldn’t be.

Show threadBenjamin Sonntag-King 🐙Aug 14, 2024
@inthehands mandatory picture from @TechConnectify <3
Show threadohmrunAug 14, 2024
@inthehands I thought everyone knew that agile stands for Avoid Grief from Investors Looking for Earnings
Show threadtiny_mAug 14, 2024
@inthehands I once sent a PR back to a colleague with some change requests because he'd missed a bunch of the acceptance criteria from the ticket and he told me in all seriousness that his work couldn't be wrong because he'd written unit tests and they were all passing and he honestly couldn't see the difference between building the thing we were meant to be making and building this other thing he'd actually made, regardless of how well he'd made it. I felt bad for him
Show threadPaul CantrellAug 14, 2024
@tiny_m
Oof, yeah. I mean, a lot of what I’m talking about falls to a product manager or program manager type of role, somebody whose job is specifically looking at the long goal of the project and thinking about •who• it’s for — but that responsibility truly extends to the whole team, right down the the dev writing a unit test (or not thinking to write one) and asking “Why this? Does this make sense? What am I missing? Is this playing out the way others think it’s going to?”
Show threadtiny_mAug 14, 2024
@inthehands I've found the only way I can be really happy as a developer is working in really tiny teams of people who all get it. I frequently hear my colleagues asking "what problem are we trying to solve" and that makes me so happy. I've tried working in bigger teams and bigger companies with roadmaps and agile and processes and raci matrices. One of my recent horror stories was being asked by an architecture committee to write an RFC explaining why we needed to change some CSS. So painful
Show threadPaul CantrellAug 14, 2024

@tiny_m
Ugh, yeah.

I’d posit that with almost no exceptions good software work happens on small teams. That’s true even in large orgs: the good ones know how to set up small teams for success even in the context of a much larger sea of people.

Show threadtiny_mAug 15, 2024
@inthehands I felt bad for them too. They'd convinced themselves if they perfected the magical formula of processes it would remove all the friction. I am not against agile, I love the underlying concept of encouraging people to talk more. Unfortunately it's too easy to fall into the habit of performing the rituals without ever questioning how effective they are or whether they are improving communication. The root of the problem was the people not their process but they couldn't see it
Show threadThe Long TailAug 14, 2024
@inthehands You get what you measure.
But you get only what you measure, because now you're measuring, nothing else matters.
Show threadPaul CantrellAug 14, 2024
@sleepyfox
Similar thought: https://hachyderm.io/@inthehands/112247731268850864
Paul Cantrell (@[email protected])

Two bitter truths: 1. You can’t manage what you don’t measure. 2. You can’t measure the thing you want to manage. (Why 2? Because you’re never, ever measuring exactly what you think you’re measuring.)

Hachyderm.io
Show threadThe Long TailAug 14, 2024
@inthehands "a costly myth"
Show threadmorganAug 14, 2024
@inthehands
Show threadAnthony SoraceAug 14, 2024
@inthehands @aparrish But look, we were like 92% of the way there! So many successful milestones!
Show threadNew-Cleckit DominieAug 14, 2024
@inthehands *nods so hard in university that my neck hurts*
Show threadPaul CantrellAug 14, 2024
@ncdominie
One of my Paul refrains is that human orgs all have essentially the same family of problems, and people get way too hung up on industry vs nonprofit vs gov vs academia etc. Different dishes, but cooked with the same ingredients.
Show threadCarl MuckenhouptAug 15, 2024
@inthehands The industry has come up with a way to address this in recent years: creating software that isn't intended to solve any problems