Mastodawn

William Pietri Jan 14, 2024

I think this, a discussion of the parallels between "AI" and "crypto", is a good take. I want to dig into the bit on "AI" being different because it has practical use.

"AI" is a marketing term. There's the stuff that was mainly called "ML" up until 2021 or so, which definitely has practical uses. E.g., if you're running a social network and need to help humans find the toxic stuff, ML can help.

But in the last few years there's a wave of hype mainly around the large language models, LLMs, and the large text-to-image models. So things like ChatGPT and DALL-E. It's really not clear to me those have much more practical use than crypto. Certainly not over their costs. 1/

https://sfba.social/@[email protected]ial/111754701923644674

Jesse Baer 🔥 (@[email protected])

People like to say that "AI" is different from crypto in that there are actual useful applications, and that's true. But the vast majority of people you're expecting to come up with those applications are the same people who were just trying to build products on the blockchain.

Mastodon

Just to be sure I'm not being unfair, I searched for writeups of LLM uses. Here's a representative example of the genre: https://www.techopedia.com/12-practical-large-language-model-llm-applications

They mention 12 use cases. Some are done better with simpler, cheaper models or non-ML techniques (1, 4, 6, 8, 12). Some are wildly speculative (2, 9, 10).

But that leaves 4 items that I want look at carefully: content creation, customer support, sales automation, and writing code. Those are at least superficially plausible places where LLMs and large image models could have practical uses.

2/

William Pietri Jan 14, 2024

First, though an important philosophical point: LLMs are fancy autocomplete. You give them a set of words, they'll predict the next word based on the enormous corpuses they've been trained on. This can give them the appearance of sentience. People will talk as if they "understand" things. They don't. It's the million-monkeys-with-typewriters thing, but the monkeys have seen enough English text that the next word is statistically consistent with the previous words.

Humans are subject to to pareidolia, and we really like to anthropomorphize things. It's not just thunder, it's a guy with a name and a look and a personality and a whole family. It's not just a bit of winter where we celebrate with our dearest; it's a fat guy in a red suit with specific facial hair. So although the text can feel human, we'll have to work hard to think of ChatGPT as a bit of unfeeling machinery, not our plastic pal who's fun to be with.

3/

William Pietri Jan 14, 2024

Ok, so first, content creation. That seems positive, right? Wrong! The best way I've seen of explaining this: "Why should I take the time to read something nobody took the time to write?"

I think this one is a huge net societal negative. The people out there who want instant "content" are almost entirely not readers. They're people who want something to run ads against. They're people who want the credit for writing without doing the work. They're people who want to sell you something without understanding the something or whether or not it might be good for you. In short, they're people with various levels of contempt for their readers.

4/

William Pietri Jan 14, 2024

As a writer, I think the real value of writing is the thinking and care that goes into it. Even for purely factual material, writing involves a careful search for truth.

LLMs, though, don't have any concept of "true". They can't. What they have is the digested correspondence between words in Wikipedia and Reddit and a zillion other sources of text. Truth can be represented in text, but it lies outside of it.

In philosopher Harry Frankfurt's "On Bullshit", he defines it as "speech intended to persuade without regard for truth": https://en.wikipedia.org/wiki/On_Bullshit

Marketing content generated by LLMs is clearly bullshit. But I'd argue that by imitating human forms of writing, *everything* produced by an LLM is bullshit. (Which would make the enormous "AI" hype cycle bullshit about bullshit, a truly American accomplishment.)

5/

On Bullshit - Wikipedia

William Pietri Jan 14, 2024

Let's turn to the second plausible use case: customer support. People often like to talk to other people to resolve problems. What if we can automate the *feeling* of talking to a person, but with no actual people involved?

There are a bunch of things going on here. At least here, there's a real user need. But to what extent is it a real user solution?

It's plausible to me here that this will be a plausible first-query solution once you've built up a good base of Q&A examples for it, doing a bit of textual generalization. At least as long as what you're doing is pretty standard. But remember that we're using a bullshit engine here, so what happens when the statistically plausible text isn't correct or useful?

6/

William Pietri Jan 14, 2024

A good example here is the car dealership that tried using a ChatGPT bot for customer service. It quickly agreed to a "legally binding" offer to sell cars for $1: https://venturebeat.com/ai/a-chevy-for-1-car-dealer-chatbots-show-perils-of-ai-for-customer-service/

Would any human agent do this? No. Because humans understand things. This is a toy example, but if GPT-ish things don't work for very basic cases, how much can we rely on them for important cases?

7/

A Chevy for $1? Car dealer chatbots show perils of AI for customer service

Incidents at car dealers highlight the responsibility of ensuring target chatbot deployment and safety compliance.

VentureBeat

William Pietri Jan 14, 2024

Moreover, I think there's a deeper problem here. Anybody who's ever tried to use large-company customer service (and I'm thinking of Amazon here) might say, "well a chatbot couldn't do much worse; it's not like talking to a person right now."

There's a really handy concept called "failure demand". Some load on a system is "value demand". E.g., I walk up to the counter in the local store and say, "I'd like a quart of milk." But if have to come back later because they're out of milk or the milk was spoiled, that's "failure demand".

For digital products, I'd argue most customer service demand is failure demand. So generally we shouldn't be putting ChatGPT in to tell people how to solve their problem, we should giving them good experiences in the first place. Chat's a bandaid.

And bad customer service generates even more failure demand. So ChatGPT might lower costs, but it won't solve the root problems or on net improve things for customers.

8/

William Pietri Jan 14, 2024

Ok, what's next? Right, "sales automation". This is a bit of a mix of the first two cases. "Sales" is a little bit customer support, and a lot of manipulation to get people to buy things. Setting aside the possible minor customer-support improvements, I think we're mostly back at content generation with contempt for the reader.

One thing people have a hard time grasping is how much our economy spends on manipulating people to get money out of them. Advertising alone is hundreds of billions. Sales is surely at least that much. And then there's all the time lost to fending off the manipulation, plus the money lost when people fall for it. On the order of magnitude of the defense budget or all K-12 spending for sure.

So a bullshit machine might be a good fit, but is this a hole we really need to dig deeper?

9/

William Pietri Jan 14, 2024

And lastly, we have writing code.

I've been writing code since I was 12. I started writing code for money at 18. And my dad started making his living as a programmer in the 1960s. And this all has a familiar ring to me.

As a young teen, so maybe 1982, I went to my first tech conference. Walking the exhibit floor, I found somebody selling a system that promised to eliminate expensive programmers by letting the business people just write in plain English.

It's an old dream, and one I've seen come up many times. Visual programming systems. Code wizards. Model-Driven Architecture. Probably many waves I've missed.

10/

William Pietri Jan 14, 2024

Of course, we also have the dream appearing regularly in science fiction. HAL 9000 is from 1968, and was set in 2001. You can just tell the computer what to do, and off it goes, understanding your needs.

I should say that this is a lovely dream. And it can be a great inspiration as we get machines and computers to do the drudgery. But as with all powerful dreams, we have to be careful to be realistic.

11/

William Pietri Jan 14, 2024

When people expect "AI" to do the coding, I think they're not paying much attention to what the hard parts of software development really are.

The computer I'm typing on is at least a million times better than where I started. A million times faster, a million times bigger. So much of the work I did then has been automated. But the job hasn't gotten easier, it's just a different kind of hard. Instead of me hacking away solo in my dad's basement, I now am trying to collaboratively build lasting, coherent intellectual works directly with my team and in concert with thousands of other people via libraries, services, and the like.

12/

William Pietri Jan 14, 2024

If I need to knock out a little boilerplate in a language I don't understand? Sure, I will happily use a bullshit generator for a chickenshit job.

But if I need to build anything that lasts, then I'm going to have to do the work myself, because the major part of the work is in *understanding* the situation (the users, their needs, the team, the history, the code), and carefully improving it, not in just typing shit out.

Is it possible that tools can accelerate this? Sure. I've been using an IDE as brain augmentation for 20+ years. I think it would be amazing if I could automate myself out of a job. But at best here I think we'll see some improved autocomplete, and at worst I think we'll see, as we have before, people generating absolute reams of garbage code that other people then have to maintain and clean up.

13/

William Pietri Jan 14, 2024

So in sum, I think the currently expected positive uses of "AI" are either things ML was already doing or possible minor improvements.

But a bunch of those improvements are on top of systems that should be rebuilt to be radically better, not just slapping another layer of shiny wallpaper on top. And a lot of what people are going to do with the ability to generate "content" is a net societal negative. Manipulation, disinformation, propaganda, and a general sea of bullshit that real humans will waste time reading.

Alas, as we've seen with the 10-year arc of "crypto", we appear to be willing to waste approximately infinite time and money as long as a) there's a hard-to-refute technoutopian gloss on top, and b) somebody somewhere is pocketing some of that money.

14/

William Pietri Jan 14, 2024

And in closing, let me recommend Zac Bissonnette's 2015 book, "The Great Beanie Baby Bubble: Mass Delusion and the Dark Side of Cute". It does a really good job laying out how bubbles work. It's a fun read and it will make you more able to spot the pieces of a hype cycle as it gets into motion.

15/15 (for now, at least)

Damon C. Jones Jan 14, 2024

@williampietri adding to the list!

William Pietri Jan 14, 2024

@damoncjones It's so good. I read it on a lark, but it's surprising to me how fresh it seemed given the "crypto" and "AI" bubbles.

Damon C. Jones Jan 17, 2024

@williampietri same vein, I recommend Attack of the 50 foot block chain. Came out a few years ago, still relevant.

William Pietri Jan 17, 2024

@damoncjones Totally agreed! I'm a big fan of @davidgerard's blog, too!

Damon C. Jones Jan 17, 2024

@williampietri @davidgerard thank you for providing his Mastodon link, I totally forgot to look him up. Great writing.

David Gerard Jan 19, 2024

@damoncjones @williampietri 🍻

Michał "rysiek" Woźniak · 🇺🇦Jan 14, 2024

@williampietri fantastic thread, thank you for putting in the time.

Jon Jan 15, 2024

@williampietri A very nice thread, I had to take notes to keep everything straight on the 12 uses.

I guess I have two reactions:

1. I loved your use of "contempt for readers". I think that is very well put on "content generation", and I haven't seen put quite that way before- it seems right.

2. While I agree with the characterization of LLMs, I think the tools we are consuming, be they GitHub-Copilot or ChatGPT, etc, are growing as software systems beyond the LLM which sits at the core of them. It could be this is all cruft which topples the value of the product, or it could be that they'll nail key quality or safety considerations and reduce error/hallucination failures to an acceptable level (whatever that means),

Anyhoo, while the "spicy autocomplete" take is one I align with, I can't help but wonder if the software systems will eventually transcend those limitations of the underlying LLM.

William Pietri Jan 17, 2024

@jjlupa Yeah, it's plausible to me that something like ChatGPT becomes the "mouth" of a richer system eventually. Sort of like the head PR person for a big company, its only job to be glib. But it's also plausible to me that it's an evolutionary dead end, too tangled and opaque to be a good foundation.

LambdaDuck Jan 19, 2024

@williampietri i find it interesting that the word “bubble” is used for something that is so obviously a deliberate scam (like BBs). but maybe that’s the case for most bubbles, just less transparently so if you don’t have insight in the specific area the bubble is in?

William Pietri Jan 19, 2024

@LambdaDuck One of the things I think is interesting about BBs is that they didn't start out as a deliberate scam. The guy who made them was an obsessive about stuffed animals. He loved them! The initial people who bought them just liked them too. But the whole thing took on a life of its own for all of them. So I think bubble is the right word there.

That definitely happens, though. I think original cryptocurrency enthusiasts were sincere. But the scammers got in early and have been making great money ever since. But even there it can be hard to tell. A lot of Ponzi scams are just people who thought something made sense, got in over their heads, and then did whatever they could to not drown. FTX was like that too; they started out to be a legit business.

Andreas K Jan 14, 2024

@williampietri
Let me summarize.

What makes me 🤮 is the hype that makes it necessary, it seems to stick AI onto anything.

When we did many of the things we do today, years ago, we just said the computer did it via an algorithm.

In our company we have been using text classification (ML) for over a decade now. Doing some other very interesting NLP.

Somehow nobody thought about selling it as AI, and beware our data science dept are evil geniuses that will have an AGI next year.

Andreas K Jan 14, 2024

@williampietri
E.g. So AI now can optimize algorithms. Headlines actually have AI write algorithms. Reality is more like AI play games that as side effect result in working implementations. Play the game better, get a better implementation.

What a revolutionary idea, gamification.

Roughly a decade ago, PostgreSQL added SQL execution optimization via a genetic optimizer. Let the best execution plan to survive the arena be used for real. They didn't market it as AI either.

Andreas K Jan 14, 2024

@williampietri By OpenAI standards how AGI is around the corner, current PostgreSQL servers should have already taken control of most satellites around Earth and Skynet is already up and running, it was just so nice not to let us know yet.

Garrett Latimer Jan 28, 2024

@williampietri Not just time and money...real energy and materials. Enough electrical demand to make old coal peakers economical. E-waste output at the rate of a decent-sized country. Aquifers drained for cooling.

@williampietri I think a good example of the bullshit case is people using AI powered tools to "find" exploits and bulk report them to try and earn bug bounties or whatever... meanwhile they're overwhelming small developers and security teams with false positives.

I'm separately curious about code explaining... but again that might work at class or library level but seems much less likely to work at system level where I usually operate.

William Pietri Jan 14, 2024

@ultranurd Yeah, agreed. I'm especially skeptical about it explaining code. Often small differences result in deep changes in meaning. And "meaning" requires coherence, which larger systems often lack. After I spend some time with a new code base, often the "meaning" is in terms of the history or the people involved.

E.g., I was brought in to help with a system that had extensive use of an early version of Docker with custom patches. It was hard to puzzle out the meaning until I found that the person who had put it in not long after did a conference talk on the fabulous new possibilities of Docker, so he'd shoehorned it into a production system and then quickly got a higher-paying job elsewhere. I ripped it all out and replaced it with something much more suitable.

Sevoris Jan 15, 2024

@williampietri @ultranurd What‘s going to be really fun is, what happens as languages get patched? Especially many of the up-and-coming languages still have (minor) semantic changes and extensions of syntax. What happens then? Is an LLM going to borrow incorrect semantics from another language because the syntax looks similar enough?

@williampietri I think the way I think about it, and something I even talk about with me peers in the context of contracts between systems, is syntax versus semantics.

As sparkling autocomplete an LLM on code isn't going to have any meaning around intent/purpose of code beyond how humans label it, just pure what's likely to come next... so boilerplate maybe helps but any business logic beyond that I'm doubtful. And how much of that could be handled just as well by better library encapsulation?

William Pietri Jan 15, 2024

@ultranurd For sure! The way I look at it, any boilerplate is duplication, an opportunity to DRY something up. Copy-paste solutions, whether direct or chatbot-laundered, are a missed opportunity for better abstractions.

LambdaDuck Jan 19, 2024

@williampietri i guess programming is an area where fancy autocomplete can be slightly useful, as long as you view it as exactly that.

as long as you know what you want and can understand the code it generates, it can be a way to save time and reduce the number of roundtrips to example code and reference documentation (assuming you aren’t missing some important caveats in the docs, but that’s a risk when using ordinary autocomplete too)

Richard Gaywood Jan 19, 2024

@williampietri My most optimistic/hopeful take is an LLM as a pair programmer & rubber ducker. At some point I want to try the free trial of CoPilot in VSCode to see how it works for that. Or maybe the JetBrains one in WebStorm.

Andreas K Jan 14, 2024

@williampietri
Ah and from time to time, I lived quite well (even if the work was painful) from refactoring and fixing systems created by non-computer scientists (sometimes even with tools extra for them, no I'm not looking in the direction of the Access team in Redmond).

I mean, who needs to know anything about relational databases when designing SQL databases. And when customers call and complain that their historical engagement numbers go down, 🤷 did the company save money with that design.

Jack Coates 🐀Jan 14, 2024

@williampietri very thorough takedown! I've taken some stabs at the code development part as well (http://monkeynoodle.org/2023/05/07/get-the-developer-out-of-the-way/) but I like your addressing of customer support as failure demand

Get the developer out of the way!

I’d be so much more productive if I could get rid of this other person who keeps asking hard questions about exactly what I mean and exactly what I expect! Just make the spaceship go to Mars, okay?…

Monkeynoodle.Org

The Laughing Muse Jan 14, 2024

@williampietri "Low code solutions"?

)}]Jan 15, 2024

@williampietri
The software that you saw in 1982 could have been the confidently-named "The Last One".

It was is released in 1981, and was created by bankrupt former millionaire (with only a week's formal training in computers) and a tyre dealer.

All of the references are worth a read:
https://en.m.wikipedia.org/wiki/The_Last_One_(software)

The Last One (software) - Wikipedia

William Pietri Jan 15, 2024

@dec23k Oh wow, that could very well be it. And you're right, the references were a gold mine. 40 years later and it's all the same as the current "AI will replace programmers" thing. The same hype, the same credulous media interest, the same questions, and the same failure modes.

William Pietri Jan 15, 2024

@dec23k And this magazine cover! We see this same media dynamic 40 years later! "The end of programming" is always just around the corner.

James Young Jan 16, 2024

@williampietri @dec23k oh look, the Internet Archive has a copy https://archive.org/details/143_The_Last_One

143 The Last One : Free Download, Borrow, and Streaming : Internet Archive

143 The Last One

Internet Archive

William Pietri Jan 16, 2024

@pronoiac @dec23k

Oh wow! Feb '82! At a quick glance, my job does not seem to be at risk.

James Young Jan 16, 2024

@williampietri @dec23k but what about the Apple BASIC programmers?

soaproot Jan 14, 2024

@williampietri A lot of customer support which isn't failure demand is feature requests, or corner cases, or other things not especially amenable to automation.

Mark Hurst Jan 19, 2024

@williampietri this is so true. I wrote an entire book on customer experience - https://customersincluded.com - encouraging teams to focus on the experience FIRST, before building out all sorts of other tech. Just a few teams listened :)

Customers Included (2nd Edition): customer experience book by Mark Hurst

Sheldon Rampton Jan 15, 2024

@williampietri The $1 car offer was NOT legally binding. Human beings hallucinate too.

Sheldon Rampton Jan 15, 2024

@williampietri https://www.businessinsider.com/car-dealership-chevrolet-chatbot-chatgpt-pranks-chevy-2023-12?amp

A car dealership added an AI chatbot to its site. Then all hell broke loose.

Pranksters figured out they could use the ChatGPT-powered bot on a local Chevrolet dealer site to do more than just talk about cars.

Business Insider

William Pietri Jan 15, 2024

@sheldonrampton Yes, I understand that. Chatbots can't legally agree to anything.

William Pietri Jan 15, 2024

@sheldonrampton What I'm referring to was well explained in the article I linked. Here's the screenshot that includes the phrase you're troubled by.

Sheldon Rampton Jan 15, 2024

@williampietri YOU wrote that the offer was legally binding. It wasn’t. The screenshot shows that phrase, but that doesn’t change the fact.

William Pietri Jan 15, 2024

@sheldonrampton I think there's another way to read what I wrote, and I'm skeptical that anybody, you included, misunderstood what was going on. But just in case, I have put the phrase that bothers you in quotes.

Next time you've got an upwelling of reply-guy energy, I suggest you start out assuming that the person is not an idiot. It might go better for all concerned.

Jenny Wu 邬珍妮 🇺🇸🇺🇦 🏳️‍🌈🏳️‍⚧️🌹✊🏾Jan 25, 2024

That is so cool 😎

In all seriousness, tricking a bot is innocent enough, but it’s a bit more ethically tricky when the opportunity presents itself to manipulate a human employee who has been trained to rely on AI assistance. For example overseas call center reps, or working immigrants from ESL communities.

I’m afraid we’ll see a lot more of that in the future.

Jenny Wu 邬珍妮 🇺🇸🇺🇦 🏳️‍🌈🏳️‍⚧️🌹✊🏾Jan 25, 2024

The discourse over “what if robots had human rights” becomes increasingly plausible, not because they pass the Turing Test, but because they don’t. It’s just easier for authorities to criminalize and punish end users who adversarially prompt AI.

So many policy makers seem driven to pick the stupidest possible solution to a problem.

Spocko Jan 14, 2024

@williampietri This is a new way of this king (Ha! Autocorrect changed thinking to 'this king' based on my previous texts! Driving home your earlier point about AI content) I had not considered about the AI "Content* and who wants it. .
"They're people who want something to run ads against. They're people who want the credit for writing without doing the work."

Sevoris Jan 14, 2024

@spocko @williampietri Oke neat way I have seen the consequences of "content-ification" framed is that the content of the content stops mattering. At it‘s best it is modular, rapidly swapable, appears and vanishes as quickly as it optimizes some marketing algorithm curve. How thoughtful or reflective or emotional or evocative it is, what philosophy it contains… don‘t matter. Culture and philosophy are at best functionalized as getting people receptive for your adware.

Sevoris Jan 14, 2024

@spocko @williampietri It was an article by @clive that formulated this basis for my observation.

Clive Thompson Jan 14, 2024

@spocko @williampietri @Sevoris

🤘🏻🤖 aha glad you liked that one!

Sevoris Jan 14, 2024

@clive @spocko @williampietri it has stuck around with me! TBH I think it has gained in relevance as further evaluation of language models has revealed how their statistic view of language pulls ideas and writing towards a washy common denominator and then the NYT lawsuit came along and demonstrated just how deep the overfitting goes...

Language models to me now look like "content machines" with all the absolute issues thereunto...

Dud (Mx)Jan 14, 2024

@williampietri "Stop, cock!" (Stopcock? I'll get my hat...)