Paul CantrellDec 4

Fascinating story from a software dev Fedi friend, shared with permission to keep it anonymous:

❝A couple of days ago, I had an experience at work that made me understand one of the reasons why the chasm of opinion about LLMs is so deep and wide.

My department mostly does fiddly lowlevel work, [close to hardware]. A few of us don't use LLMs at all, a few use them sparingly, and one member is absolutely all-in. So during one of our morning meetings he suddenly started going off on a deeply disturbing diatribe about how we need to treat the LLMs “like slaves”.❞

1/

Show threadPaul CantrellDec 4

❝I said that I *absolutely in no way* want a slave, or a technology that simulates one. I want to do creative work using good tools. I *don't want* the experience of a slaver; in fact, I would go very far to never have that experience, because it is a demeaning and antihuman experience.❞

2/

Show threadPaul CantrellDec 4

❝He was completely unable to understand why. He kept arguing that since the LLM isn't an actual conscious person (which is correct), it ought to be treated like a slave, and that the arc of technology is to give everyone access to their own virtual slaves.❞

3/

Show threadPaul CantrellDec 4

❝It was *impossible* for me or anyone else present to get him to understand that *we don't want slaves, simulated or otherwise*.

I've thought about that a lot in the days since.❞

4/

Show threadPaul CantrellDec 4

I agree with the storyteller: the experience of having a slave is abhorrent, simulated or not, and this story is a window into something •deep• about the present moment. Even without having slaves, we are all in danger of having a •slaver mindset•. It’s a disease that’s running rampant now in billionaire-shaped techno-utopian circles.

I wrote this thread on the topic earlier:

https://hachyderm.io/@inthehands/113295613785073188

…and even having written that, it’s still shocking — not surprising, exactly, but shocking — to hear those thoughts expressed so baldly by the colleague in the story above.

Paul Cantrell (@[email protected])

@[email protected] @[email protected] @[email protected] Guinan said it. This scene still gives me chills every time: https://www.youtube.com/watch?v=-T9TUeapBSQ What writers. What a pair of actors.

Hachyderm.io
Show threadOlga Lovick (she/her)
@inthehands I have felt like that for a long time about Siri and Alexa! (And the fact that both are gendered as women makes it even harder.)
Dec 4 at 7:04pmWeb
Show threadMx. Aria StewartDec 5

@transitionalaspect @inthehands Oh I have _feelings_ about this, especially American Voice 5.

See the default for American Siri is female — subordinate, idealized feminine subservience. You can change it to a male voice, of course, and it is in some regions. Not everyone has the same relationship to being served and who feels safe and trusted with those tasks — in some cultures that might be men, of another class or nationality — that makes people feel right in their place in the world. But in the US, quite often it's a not-quite-Mom, not-quite-girlfriend, maybe-a-secretary voice that makes us feel at ease.

But of course not for everyone. We need to represent the culture we're in to really hit it with everyone. A lot of Americans rightly notice that it's rather binary, man or woman. So Apple introduces American Voice 5. An explicitly nonbinary voice. It feels very natural and friendly, just like a helpful person who’ll give you just what you ask for.

Where is that training of us coming from? Why does that sound so reassuringly servile, unthreatening, invisible?

Try your local coffeeshop. Listen to the staff.

So many of them sound just like American Voice 5.

Show threadalphaDec 5
@aredridel I did see an interesting video short recently (I forget which service) on how AI voices have changed through time, as a reflection of how our culture thought of AI.
Show threadMx. Aria StewartDec 5
@alpha Yes! I find the sociology of it fascinating, even if the end result i find truly unnerving.
Show threadOlga Lovick (she/her)Dec 5
@aredridel *shudders* @inthehands