Prof. Emily M. Bender(she/her)

Veatch: “What was really surprising to me during my initial dive into all of this was how, when you look at the question of superintelligence as a documentarian or journalist, it doesn’t take long before you smack your forehead into the low doorframe of race science, because it’s baked into this technology”

I love this metaphor because of how it speaks to the contortions people (here: AI boosters) go through to ignore/pretend not to notice the race science.

https://www.theverge.com/entertainment/897923/ghost-in-the-machine-valerie-veatch-interview

The gen AI Kool-Aid tastes like eugenics

Ghost in the Machine — out on Kinema March 26th — director Valeria Veatch speaks with The Verge about gen AI’s roots in eugenics.

The Verge
Mar 21 at 2:39pmWeb
Show threadCurtAdams1d ago
@emilymbender Is there a better term than "race science"? 'Cause it sure ain't science.
Show threadBlake C. Stacey1d ago
@CurtAdams @emilymbender I sometimes say "Numberwang racism".
Show threadBrad Macpherson23h ago
@bstacey @CurtAdams @emilymbender Bonus points there for the visual similarity between Numberwang and Nuremberg (to me at least) 👍
Show threadJoan of Contention 😷19h ago

@CurtAdams

Pseudo-science.

@emilymbender

Show threadsome kind of orange shape58m ago
@clickhere @CurtAdams @emilymbender Race science is bad science, but I think excluding it from the category of science entirely only helps other scientists feel better. Any time a social scientist uses IQ for anything, their science is bad in the same way.
Show threadViolet/Multi1h ago
@CurtAdams @emilymbender TESCREALism! unfortunately
TESCREALism: The Acronym Behind Our Wildest AI Dreams and Nightmares

To understand the divide between AI boosters and doomers, one must unpack their common origins in a bundle of ideologies known as TESCREAL.

Truthdig
Show threadTheodora Ward1d ago
@emilymbender i still cannot believe the original "sparks of agi" paper's citation for the definition of intelligence. i mean, i can believe it, but it's stunningly obscene in a way i feel like everyone should mention at the top of every single piece anyone runs about any of this stuff
Show threadquixote1d ago

@emilymbender We really need to start calling it Autocomplete, because that's what it is. (Based on its training, it slots in the likeliest next step. That's autocomplete.)

So it replicates the misogyny, racism, bigotry of the datasets it was trained on.

If the techbros running that nonsense didn't just throw the whole web into the training set, if we had, say, reference librarians suggesting useful works to use, we could have a very different Autocomplete.

Show threadProf. Emily M. Bender(she/her)1d ago
@quixote are you perhaps unfamiliar with my work?
Show threadquixote1d ago

@emilymbender A little bit. I read your comments here and sometimes any links. From that smallish sample size, I really like your thinking on AI, um, Autocomplete.

My comment wasn't directed at you so much as shouting at the clouds. At least in my mind, I'm agreeing with you (besides shouting at clouds).

Show threadProf. Emily M. Bender(she/her)1d ago
@quixote Hey, if you're shouting at the clouds, then @ the clouds, not me.
Show threadgrrl_aex1d ago

@emilymbender

Being eugenic-adjacent is just one demented poison in ai's well.

Human cloning, done digitally, is another. And that our *minds* are what is being cloned just makes it even more deplorable and sick.

Show threadPrem Kumar Aparanji 👶🤖🐘19h ago
@kitkat_blue wouldn't a homunculus be better than clone? @emilymbender
Show threadspdrnl1d ago
@emilymbender After ad-tech and social media, AI will wind up as the next frontier in tech that dictates you how to think.
Show threadspdrnl1d ago

@emilymbender The worrisome part is that that is probably more then enough incentive to finance the shenanigans.

Best of all, contrary to social media, people will pay to get indoctrinated.

Moguls are having a field day.

Show threadMarty Fouts23h ago
@emilymbender perhaps “race pseudoscience” is a better term?
Show threadG. Clavier13h ago
@MartyFouts @emilymbender Definitely.
Show threadAmin Girasol18h ago

@emilymbender this is very revealing about how access to power works:

When I asked Veatch if she had ever been interested in speaking directly with the heads of the companies Ghost in the Machine takes to task, she laughed. Getting that kind of access, she said, would require her to go through all kinds of ideological gymnastics and make compromises that would make her film complicit in gen AI’s harms.

Show threadGhostOnTheHalfShell11h ago

@emilymbender

I watched Taylor Lorenz’s hour long discourse on the culture of tech bros, which I think applies more widely to most billionaires. She called them an extinction cult. “ conservative “ economic ideology legitimizes & valorizes dark triad personality traits. Businesses deliberately dehumanize everyone, customers, employees. they don’t respect the boundaries people demand.

Their goal is to replace humanity. That one on its own explains indifference or even worse

Show threadAaron5h ago

@emilymbender Can you point me to something to read on how the race science influences the technology and science of AI?

This isn't a gotcha. I just don't get it - unless I am thinking about this wrong and it has more to do with the culture of the people developing this stuff, how the LLMs are trained and so on where the race science creeps in.

I have tons of criticisms of AI but they are primarily linked to who is funding this and why not the roots of the tech itself.

Show threadProf. Emily M. Bender(she/her)5h ago
@subterfugue Watch Ghost in the Machine, i.e. the film (available through Kinema this coming weekend!) that the article in my op is about.
Show threadAaron5h ago

@emilymbender I understand you pushing me off toward what you posted given that this is the internet and comments like mine are usually a misrepresentation and a trap. But I am actually genuine here, and I don't have much interest in that documentary after reading her interview which seemed quite shallow.

I think there is something to the history of this, but I would rather read something in depth to understand what the connection and influence actually is rather than the guilt by association that she is relying on.

I appreciate the work that you are doing and wish you the best.

Show threadProf. Emily M. Bender(she/her)5h ago

@subterfugue I responded as I did because it is quite rude (I think) to reply to a post about a piece of media, not acknowledge that media, and ask a question answered in that media.

But if you prefer to read, try:

https://firstmonday.org/ojs/index.php/fm/article/view/13636

https://thecon.ai

https://www.hachettebookgroup.com/titles/adam-becker/more-everything-forever/9781541619593/

The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence | First Monday

Show threadProf. Emily M. Bender(she/her)5h ago
@subterfugue That said, your jump to "guilt by association" suggests to me that you are very resistant to the idea that "AI" is based in race science and are in fact just looking for reassurance for your own point of view.
Show threadAaron4h ago

@emilymbender Depending on what you mean by "based in race science" I am of course resistant to that.

The assumption of most people who hear that is that the tech can never be of any value because of such a taint. I completely disagree. I think the tech could potentially be of value. (I think AlphaFold is valuable for example)

If you however mean that the history of the development of this stuff has race science all over it, and that the people driving its development right now are influenced by race science or believers in it, YES I agree. Of course.

Anyway, thank you for those articles. I've been reading them for the past hour. I don't have your expertise and thus didn't find them.

Show threadltning3h ago
Thank you @emilymbender - I'll watch this when it becomes available to me.

The part "...we have to know what the fuck that phrase means" resonates: Would not current "AI" better be described as "simulated intelligence"?
As with diamonds, simulated ones may look like the real thing at a distance, but being made of plastic, glass or cheaper crystals, they cannot stand up to scrutiny and have limited usefulness. Artificial ones however are nearly indistinguishable from real ones, and are as (or more) useful.