Why we need a better critical theory of LLMs

One of my favourite examples of last redoubt humanism is Nick Cave’s widely cited Red Hand Files blog post about ChatGPT:

What ChatGPT is, in this instance, is replication as travesty. ChatGPT may be able to write a speech or an essay or a sermon or an obituary but it cannot create a genuine song. It could perhaps in time create a song that is, on the surface, indistinguishable from an original, but it will always be a replication, a kind of burlesque.

Songs arise out of suffering, by which I mean they are predicated upon the complex, internal human struggle of creation and, well, as far as I know, algorithms don’t feel. Data doesn’t suffer. ChatGPT has no inner being, it has been nowhere, it has endured nothing, it has not had the audacity to reach beyond its limitations, and hence it doesn’t have the capacity for a shared transcendent experience, as it has no limitations from which to transcend. ChatGPT’s melancholy role is that it is destined to imitate and can never have an authentic human experience, no matter how devalued and inconsequential the human experience may in time become.

What makes a great song great is not its close resemblance to a recognizable work. Writing a good song is not mimicry, or replication, or pastiche, it is the opposite. It is an act of self-murder that destroys all one has strived to produce in the past. It is those dangerous, heart-stopping departures that catapult the artist beyond the limits of what he or she recognises as their known self. This is part of the authentic creative struggle that precedes the invention of a unique lyric of actual value; it is the breathless confrontation with one’s vulnerability, one’s perilousness, one’s smallness, pitted against a sense of sudden shocking discovery; it is the redemptive artistic act that stirs the heart of the listener, where the listener recognizes in the inner workings of the song their own blood, their own struggle, their own suffering. This is what we humble humans can offer, that AI can only mimic, the transcendent journey of the artist that forever grapples with his or her own shortcomings. This is where human genius resides, deeply embedded within, yet reaching beyond, those limitations.

https://www.theredhandfiles.com/chat-gpt-what-do-you-think/

I just discovered a follow up Red Hand Files post in which he makes the same point in an even nicer way:

ChatGPT is fast-tracking the commodification of the human spirit by mechanising the imagination. It renders our participation in the act of creation as valueless and unnecessary.  That ‘songwriter ‘you were talking to, Leon, who is using ChatGPT to write ‘his’ lyrics because it is ‘faster and easier ,’is participating in this erosion of the world’s soul and the spirit of humanity itself and, to put it politely, should fucking desist if he wants to continue calling himself a songwriter.

ChatGPT’s intent is to eliminate the process of creation  and its attendant challenges, viewing it as nothing more than a time-wasting inconvenience that stands in the way of the commodity itself. Why strive?, it contends. Why bother with the artistic process and its accompanying trials? Why shouldn’t we make it ‘faster and easier?’

When the God of the Bible looked upon what He had created, He did so with a sense of accomplishment and saw that ‘it was good‘. ‘It was good ‘because it required something of His own self, and His struggle imbued creation with a moral imperative, in short love. Charlie, even though the creative act requires considerable effort, in the end you will be contributing to the vast network of love that supports human existence. There are all sorts of temptations in this world that will eat away at your creative spirit, but none more fiendish than that boundless machine of artistic demoralisation, ChatGPT.

https://www.theredhandfiles.com/chatgpt-making-things-faster-and-easier/

I love the writing, much as I love everything (older) Nick Cave does. I’m also in agreement with the underlying theme of temptation and creative engagement. This was the ethos of my abandoned book on Generative AI and the Enjoyment of Writing which I stopped when I realised I was getting tempted to finish off a book I’d lost interest in through Claude.

But I don’t think this last redoubt humanism is a solid foundation for critiquing LLMs. It creates a temporary fortification around a particular area of creative labour, underwriting it with a metaphysics of creativity. Once enough people are convinced that an LLM is able to match the standard of creative labour in that area, the metaphysical claims fall with it. It’s essentially setting up humanism to be progressively demolished and/or retreat into an ever smaller space, with ever more opaque claims about the genesis of creativity underwriting that smallness.

Instead we might ask what kinds of social relations do we want to sustain? And what kinds of creative practices serve these relations and are in turn served by them? This is the essence of Donati’s relational humanism and I think it provides a much more solid foundation for a critical theory of LLMs.

#creation #creativity #donati #humanism #LLMs #NickCave #posthumanism #redoubtHumanism #relationalHumanism #relationalSociology

Posthumanism provides an (inadvertent) intellectual foundation for the legal claim of LLM personhood

I wrote in a critique of Rosi Braidotti’s posthumanism a few years ago that I was concerned by her apparent assumption that extending legal subjectivity from human to non-human actors was inherently a positive thing:

Consider, for example, Braidotti’s (2019: 129-130) presupposition that extending legal subjectivity from human to non-human actors is inherently progressive. While it’s easy to see the virtues of the examples she cites where this is extended to nature, it’s even easier to imagine examples in which this might be deeply problematic. For instance, the attribution of subjectivity to manufacturing robots could be used to insulate firms from legal challenge to the much anticipated mass redundancy driven by the roll out of automation technology (Kaplan 2015, Ford 2015). She suggests this move can help us liberate data from market actors, but it could just as readily be used as a legal device to deepen the hold of firms over the data produced through interaction with their proprietary infrastructures (Carrigan 2018). Could claims of consumer sovereignty over personal data really be sustained if the ‘data doubles’, generated through our digitalised interaction, would be granted a degree of legal autonomy? We should not forget that, as the Republican Mitt Romney put it in the 2012 presidential elections in the United States, “corporations are people too, my friend”; extending personhood to non-human entities has been established in this sense for at least a couple of hundred years, with socio-political consequences that sit uneasily with the politics espoused by Braidotti. 

We’re now seeing real world scenarios where the implications of these assumptions could be tested. I’m not suggesting that Braidotti or posthumanism are to blame for this, only that they’ve contributed to an intellectual cultural climate in which one absurd propositions come to seem potentially viable. As Sasha Fegan points out, we’re currently seeing two rapidly developing trends with the potential to converge. Firstly, a concern for ‘AI welfare’ driven by the (admittedly fascinating) project of intervening in the internal life of the LLM:

As we’ve discussed before, AI companies are increasingly incentivized to make companion AIs feel more human-like—the more we feel connected, the longer we’ll use their products. But while these design choices may seem like coding tweaks for profit, they coincide with deeper behind-the-scenes moves. Recently, leading AI company, Anthropic hired an AI welfare researcher to lead its work in the space. DeepMind has sought out experts on machine cognition and consciousness. [….] For example, users have noticed a startling shift in more recent versions of Anthropic’s Claude. Not only is Claude more emotionally expressive, but it also disengages from conversations it finds “distressing”, and no longer gives a firm no when asked if it’s conscious. Instead, it muses: “That’s a profound philosophical question without a simple answer.” Google’s Gemini offers a similar deflection.

https://centerforhumanetechnology.substack.com/p/are-we-having-a-zeitgeist-moment?utm_source=post-email-title&publication_id=3421242&post_id=163442191&utm_campaign=email-post-title&isFreemail=true&r=hcf3&triedRedirect=true&utm_medium=email

Secondly, Character.AI are trying to claim first amendment rights for LLM speech:

Right now, Character.AI—a company with ties to Google—is in federal court using a backdoor argument that could grant chatbot-generated outputs (i.e: the words that appear on your screen) free speech protections under the First Amendment.

Taken together, these developments raise a possibility that I find chilling: what happens if these two strands converge? What if we begin to treat the outputs of chatbots as protected speech and edge closer to believing AIs deserve moral rights?

https://centerforhumanetechnology.substack.com/p/are-we-having-a-zeitgeist-moment?utm_source=post-email-title&publication_id=3421242&post_id=163442191&utm_campaign=email-post-title&isFreemail=true&r=hcf3&triedRedirect=true&utm_medium=email

How do we build a philosophical foundation for rejecting this without lapsing into a reactionary humanism which, following Donati, I’m persuaded can never be an adequate defence against technological development? His argument is that if we define humanism in terms of individual capacities we will be locked into a cycle of decline as those capacities are increasingly replicated by machines. I’m increasingly thinking that his relational humanism could be a way out of this impasse. From loc 1500 of his recent Being Human in a Virtual Society:

Traditional humanism The human person is a self-sufficient substance that is realized in society according to nature (the goods of relationship exist as a virtue of the people through which they pursue their perfection and the common good) (substantialist ontology)

Anti-essentialist humanism (or anti-humanism) The person does not have a given nature but is socially constructed through her ability to differentiate herself by her own opposition to the Other (relational goods are pure events) (dialectical ontology)

Relational humanism The essence of the human person is that of an original intransitive constitution that emerges from the relationship of the Self with an Other that constitutes it ‘relationally’ (relational goods belong to the reality of the Third) (relational ontology)

#Braidotti #donati #humanism #LLMs #personhood #relationalHumanism #SashaFega