Here is your must-read article for the day, a profile of @emilymbender, and her efforts to deflate the ridiculous hype around large language models such as ChatGPT.
It's also about the people who are behind that hype, and about what their way of thinking has the potential to do to us.
It's worth reading all the way to the end.
https://nymag.com/intelligencer/article/ai-artificial-intelligence-chatbots-emily-m-bender.html
I find it weirdly striking of how much in this debate just seems like a "affirming the consequent" fallacy - in what world is the brain is a computer => a computer is a brain sound in any way?!
Similarly, granted that a bunch of human learning is associative - that doesn't mean all of it is, or that any associative learning would be human
So frustrating how conversation about #LLM and #ChatGPT completely misses the most important point.
We're so easily distracted by #SciFi questions like whether it's a mind, is it sentient, will it destroy humanity.
(No, no, and maybe.)
The most important point is that we (rightly) fear AI run amok because we (rightly) fear #capitalism run amok.
Yes, it hallucinates bogus facts.
Yes, it can be tricked into acting like a creepy stalker.
No, it doesn't simply predict the statistically most likely next word in a sequence.
It actually has an internal model of concepts and relationships, and can draw meaningful and truthful insights that are useful to humans.
In many cases, it's more useful than search engines or Wikipedia.
It's a powerful tool, and like all powerful tools, it will be used for good and for evil.
And you can bet that the evil uses are currently being accelerated and amplified by billionaires and profit-seeking corporations exploiting workers and customers and subverting democracy to enrich themselves.
Same as it ever was.
That's why the real point is that we need non-profit public-benefit organizations to drive safe and positive and trusted uses of this technology.
The Internet Archive, Wikipedia and craigslist are good examples.
The genie is out of the bottle. We can't wish this technology away.
Let's ensure it gets used for good.
Let's ensure humanity has a fighting chance against capitalism amplified by intelligent machines.
@ares @ct_bergstrom @emilymbender But this is a technology that requires hundreds of millions of dollars to implement. The field is dominated by the largest monopolies on the planet. AI is an expression of pure hypercapitalism, it's impossible to divorce it from that matrix.
Our future -- artifical humans owned and operated by mega-corporations, slowly eating away at our conception of humanity.
@zenkat @ct_bergstrom @emilymbender
yes, it's bleak
it's hard to imagine how we can prevent dangerous practices by capitalists who can operate from any jurisdiction
but at least we can build a trusted alternative that won't manipulate us with disinformation to extract profits to billionaires
wikipedia also costs more than a hundred million a year to operate
i'm incredibly thankful that wikipedia and mastodon and open-source software exist in a world controlled by capital
safe and trusted non-profit alternatives to capitalist #ai should also exist, or we're truly fucked
@ares @ct_bergstrom @emilymbender > It actually has an internal model of concepts and relationships, and can draw meaningful and truthful insights that are useful to humans.
The nature of that model also matters. It wouldn't be making trivial arithmetic mistakes if that model included the semantics of its information.
@ares
Do you have a reference on:
> No, it doesn't simply predict the statistically most likely next word in a sequence.
>
> It actually has an internal model of concepts and relationships, and can draw meaningful and truthful insights that are useful to humans.
From reading the recent Steven Wolfram article, my understanding was that it predicts a few statistically likely words and somewhat randomly picks one.
there's a lot of confusion about this because predicting the next word and then comparing the prediction with the actual next word in a large corpus of text is how the model is *trained* and not how concepts and relationships are represented internally
internally every concept is a vector in high-dimensional space, which is how it distinguishes the 400+ different meanings and senses of the english word "set" for example
i'll let chatgpt itself explain it
@ct_bergstrom @emilymbender I want to be her friend.
"Bender is 49, unpretentious, stylistically practical, and extravagantly nerdy — a woman with two cats named after mathematicians who gets into debates with her husband of 22 years about whether the proper phrasing is “she doesn’t give a fuck” or “she has no fucks left to give.”"
@ct_bergstrom @emilymbender It is a good read, although I don't see it all the same way as @emilymbender does.
I also found Manning's undermining of body-language odd. It's pretty crucial in every-day social activities. You can even _hear_ body language behaviour in speech.
@ct_bergstrom @emilymbender Everyone arguing against ChatGPT does not understand the purpose of ChatGPT.
It is not to be good.
It's a massive misinformation engine **and that's the feature, not the bug**.
Having linguists and others telling everyone that ChatGPT is a massive misinformation engine is like telling people Trump is racist. It's doing their marketing for them.
@ct_bergstrom @emilymbender Would love to see this debate. Is it available somewhere?
On the highlighted part, I think it’s both. We are not a stochastic parrot. Heck, we aren’t even our brains. But our stochastic parrot brain plays a crucial role in making us who we are.
@alter_kaker Weird. I thought Manning had the better take, if those two surely oversimplified descriptions were accurate.
I’d point to people’s (mis)use of grammar as a good example of how we learn language to a large extent through observation and not through formal learning.
@alter_kaker I think that’s also a good example of the problem with relying *solely* on this inductive form of reasoning (which are the same problems with LLMs).
As I said elsewhere, though, ChatGPT et. al. are impressive compared to a child their age.
@alter_kaker Wow, I completely disagree. What learning do children do “every second” that isn’t inductive? Learning through testing boundaries *is* inductive. Quintessentially so. Learning through social feedback is generally inductive. And kids learn as much if not more (probably more) by copying what they see others do than through social feedback.
Perhaps the big difference, which Manning also points out, is that kids learn through multi-modal input. That makes a huge difference.
The first time my son (then just a few years old) went through a tunnel (in our car), two police cars came through the tunnel, passing everyone, flashing their lights.
On the way home we went through the tunnel again. I could see him looking around for the police cars.
I dunno. Seems a lot like self-supervised learning to me. But maybe I’m misunderstanding the debate, which is why I was interested in seeing a transcript or video of it.
@anthony @ct_bergstrom @emilymbender could it be this?
@escapadesrpg The article describes the singularity as “the tech fantasy that, at some point soon, the distinction between human and machine will collapse.”
Umm, no, the singularity would very much not involve machines becoming a thing even remotely close to humans.
But I’m not sure journalists are ruining journalism so much as that these sorts of ignorant, biased articles have always been part of journalism. Critical reading has always been an important part of reading all kinds of text.
"...chatbots that we easily confuse with humans are not just cute or unnerving. They sit on a bright line. Obscuring that line and blurring — bullshitting — what’s human and what’s not has the power to unravel society."
Do chatbots dream of electric parrots?
I see two major and different types of personalities responding here: The intellectual, driven by ego, and the common sense “human” wise thinker
@ct_bergstrom @emilymbender From a serious perspective, I have just finished reading that article and am so glad you brought it up and linked it. I found it very interesting and helpful.
Less seriously and going back to my original reply, I may or may not be an anthropomorphic parrot, but I am not a stochastic parrot or a stochastic human.
@STCmicrobeblog @ct_bergstrom @emilymbender
Speak for yourself
I'm just a pretrained transformer at the tail end of a reinforcement learning process with human feedback who happens to share many of the same referents as my trainers
@ct_bergstrom @emilymbender @vampiress
Poorly cited article. It introduces Emily in relation to her recent 2020 publication, but fails to cite it. For those who are interested, I think this is it:
@ct_bergstrom @emilymbender I was remembering "signifier and signified". One thing that's bemusing me is the tech cult just directly rejecting the idea that there is anything signified beyond the signifiers.
I was dubious about the concept of qualia the first time I encountered it, but the more I've thought about it, the more important it seems to be. Qualia are the aspects of an experience that cannot be communicated, only known through direct experience.
@ct_bergstrom @emilymbender Hey that's a disservice to parrots (and cats and dogs and…) who may not agree with us on what words mean, but figure out ways to communicate with us.
In other words the parrot may now know the etimology of "Arrrr matey" but the parrot knows it means they may get a cracker if the human is in a good mood.
I work as a developer but I do not work with anything concerning what we would call "AI". I think this technology can be useful, but I'm so glad to read something with actual credibility behind it about this industry and the problems behind it and all the hype. Nothing that I have read recently about AI has talked about it realistically, this is a breath of fresh air.
Marcia Hendrick
Carl T. Bergstrom
Deborah Apthorp
leonhard volz
Alberto Cottica
Tom Dewar
IanNick
a66ey 🇪🇺🏳️🌈 she/her
ares
zenkat
Rev. Watt deFalk, O.C.D.
R. A. Dehi
LisPi
sbszine
Andrew Hewus Fresh
DrArtAnalytics
vampirdaddy
Erik Jonker
Sasha Fox
Dynom
dr 🛠️🛰️📡🎧
vasssbam
Anthony DiPierro
Marc Trius
escapadesrpg
Michael J DeLuca
mike mahaffie
Jedigal007
Ferles 
Aris Leepvogel, gelukzoeker ✅
SomersetWhovian 🇺🇦💙
John Pearce
Small Things Considered
Bornach
Elizabeth Sudduth
Joel LeBlanc
jonathanpeterson
FoolishOwl
Óscar Morales Vivó
Bradley
Gabriel N