Mignon FogartyThis story about ChatGPT causing people to have harmful delusions has mind-blowing anecdotes. It's an important, alarming read.
myrmepropagandistThis is so baffling to me. I've seen a less extreme version of this from some people ... who I thought would have known better.
What I find infuriating is how the industry selling LLMs has encouraged this kind of thinking with alarmist "AI might take over" and "here comes the singularity" talk. So irresponsible and dishonest.
That said, if you lose someone to an AI fueled spiritual fantasy I think in the absence of the tech it would have just been something else.
@futurebird Maybe it would have been something else for some people, but I saw someone say that it's like everyone who is susceptible to cult thinking now has 24-7 direct access to a cult leader, and for every person it is designed to reel them in specifically. And that felt true to me.
I had a friend who thought chat GPT was "very helpful with criticism of poetry" ... and that's true in the sense that it will look at phrases in your poem and Frankenstein together some responses with the general form and tone of what real people have said about real poems with similar patterns.
Maybe that might give someone some ideas for revisions along stereotypical lines.
It can't scan rhythm. It could easily miss the whole point and lead you to make derivative revisions.
I tried to explain why I thought it was a less than ideal method of getting feedback.
But, bumped up against a LOT of resistance. More than made sense to me.
So I decided to try it with one of my own poems.
The "feedback" was very flattering, ego-stroking in tone. Which made me really uncomfortable. I have no reason to think any real person might respond in the same way.
But I could see how if it seemed like "meaningful" feedback being told it's not wouldn't be pleasant.
Asking a LLM about your poems isn't the same as turning to it for religion... but I think it's along the same lines.
Wyatt H Knott@futurebird @grammargirl Like asking your mom. Which everyone tells you never to do.
0x@Wyatt_H_Knott @futurebird @grammargirl I implore you wonderful people to try local LLMs with an open mind. Like most powerful tools, you get out of it what you put in and it can surely be misused.
Once it's running on an aging, modest computer right next to you, it's hard to not notice the staggering implications for our relationship with society's existing power structures. I have a personal tutor right out of Star Trek that nobody can take away from me or use to spy on me. Apart from all the cool automation and human-machine interaction it opens up, for an increasing number of tasks I can completely stop using search engines.
I think the "AI is just NFTs again but more evil; it just means the inevitable acceleration of billionaire takeover and ecosystem destruction; everyone who uses it is a gullible rube at best and an enemy at worst" clickbait ecosystem is misleading and unhealthy. It fuels conspiratorial thinking, incurious stigma, and a lot of unnecessary infighting.
@necedema @Wyatt_H_Knott @grammargirl
Do “local LLMs” use training data compiled locally only? Or do you have a copy (a snapshot) of the associative matrices used by cloud based LLMs stored locally so you can run LLM prompts without an internet connection?
Aaron@futurebird @necedema @Wyatt_H_Knott @grammargirl If you have the memory & compute power on your local machine to actually run it, you can download the whole thing and run it locally, completely disconnected. It's conceivable that you could use only local training data, but good luck gathering enough local data. Also, training it would be unbelievably time-consuming if you don't use the cloud and you want anything robust. What's usually done is you fine-tune a pre-trained model on your own data, and then feed it local data and system prompts to make the responses appropriate to your use case.
@hosford42 @necedema @Wyatt_H_Knott @grammargirl
So, almost no one is using this tool in this way.
Very few are running these things locally. Fewer still creating their own (attributed, responsibly obtained) data sources. What that tells me is this isn’t about the technology that allows this kind of recomposition of data it’s about using (exploiting) the vast sea of information online in a novel way.
Philip Bragg@futurebird @hosford42 @necedema @Wyatt_H_Knott @grammargirl The assumption that it's Star Trek computer accurate with its replies once you've done this is so wide of the mark, though. Moments ago I asked Google about the formula for calculating resonance of a tuned LC circuit and two of the steps in the four step AI reply were completely pointless and didn't do what it was saying they did.
@synx508 @futurebird @necedema @Wyatt_H_Knott @grammargirl When I use LLMs, I easily spend half of the time weeding the content to identify what's actually useful. Even the best model for the task has this problem.
@hosford42 @futurebird @necedema @Wyatt_H_Knott @grammargirl They're like a rubber-ducking system, sometimes, I guess that is useful but it's possible to fool yourself into believing the intelligence you're using to hammer the machine into producing the shape of answer you're looking for is the machine's rather than yours. Clever people used to be able to have hilarious conversations with megaHAL on the same basis. Lately I've been throwing subtly incorrect statements into the Google search box to see how it would patronisingly correct me. It doesn't always do that, about 25% of the time it'll create an answer founded on the belief that I must be correct and it somehow didn't already know something - forming its own pseudo-beliefs to support my statement. I don't think this is great or useful, it is dangerous.
@hosford42 @necedema @Wyatt_H_Knott @grammargirl
It’s like the sophomoric “hack” for keeping up with all the damned five paragraph essays. Just search on the internet for a few documents with good-sounding paragraphs. Copy and paste chunks of sentences into a word document. Then carefully read and reword it all so it’s “not plagiarism”
This is still plagiarism.
GPT will cheerfully help with that last step now.
@hosford42 @necedema @Wyatt_H_Knott @grammargirl
And yet none of the English and History teachers I know are very worried about this. Because the results of this process are always C-student work. The essay has no momentum, no cohesive idea justifying its existence beyond “I needed to turn some words in”
It’s swill.
Glitzersachen@futurebird @hosford42 @necedema @Wyatt_H_Knott @grammargirl
English and History teachers have not been thinking their whole life about (a) the position of humans in a technological society and (b) the fallout from technology.
I seriously think the don't see the holocaust of culture coming.
George Girton@futurebird this sounds a lot harder than actually writing something🍸😹
Martijn Faassen@hosford42 @necedema @Wyatt_H_Knott @grammargirl
There's a whole movement of running them locally but it's niche, though the smaller versions are getting better.
But nobody is creating them locally. Fine-tuning with your own data yes, but that's a little sauce over existing training data.
Smoljaguar@futurebird @hosford42 @necedema @Wyatt_H_Knott @grammargirl if you're interested in more "ethically trained" LLMs the Allen AI institute have been doing really interesting work https://allenai.org/
