Last night, one of my students thought he had read a specific poem by a poet we're about to read, but I didn't recognize his description of the content of the poem. So he opens up Google and types in the poet's name and the topic, and it just spat out a fabricated poem in her style. This is what's really unnerving--"AI" is not adding value to a search service that works; it's flooding the search results with so much crap that you can't even verify a date or the existence of a text anymore.

In this case, no, the student was misremembering the name of a poet he read last semester. But, unable to identify a poem on that topic by Phillis Wheatley Peters, Google just made one up.

I've had problems all month trying to put the publication dates of texts next to the titles on the syllabus--quickly searching for a bunch of dates in a row, a lot of them are off by a few decades--just small enough that if you're not really thinking hard about it, might seem plausible.

@carrideen this chimes with the weird verses purportedly from Smith’s “Beachy Head” a student included in an essay last April. The right rhythm and general area of focus but strangely tautological. The student had cheated, as a painful conversation soon drew out, but my real takeaway, as yours, is that AIs are generating weird poetic content and misattributing it in ways that may force us back to scans and other facsimiles in the classroom.

@carrideen

I really hate it when people tell me that they got information from Google and cannot tell me the actual site Google sent them to that the information came from.

I try to imagine someone reading information in a book and telling everyone they got it from the card catalogue in the library.

This was happening before the the AI bullshit that only magnifies the problem by orders of magnitude.

@the5thColumnist But Google did have pretty reliable "cards" with verified information about texts and authors. I think they've stopped verifying them altogether and now info is scraped or just invented. I've tried using the function to flag incorrect info on cards and nothing has been fixed. I wouldn't be surprised if it goes to a dead email.
@carrideen @the5thColumnist Yeah, you really have to go to a recognizable site now that there is so much made up information. I block the AI result but those "cards" or whatever they are called used to be pretty reliable.
@the5thColumnist @carrideen They get the information from Google which gets it from the LLM which learned it by scraping Reddit and Wikipedia. And good luck getting them to understand why this is a problem.

@the5thColumnist @carrideen
students need to be given the same caution for using Google* that has been standard for wikipedia for two decades.

Edit* AI search in general really.

@carrideen I feel this will change over time, but if it doesn't change rapidly, we may have intrenched garbage that is hard to remove. Humans do the same thing, make stuff up and attribute it to Mark Twain or Abe Lincoln. WE do it with politicians all the time... Remember the quote from trump saying if he were to run for office he would run as a republican because they are easier to fool?? Or Hillary laughing at rape victims??? We don't need AI to lie. And we don't need AI to be fooled.
@Jon_Kramer @carrideen IMHO the problem is the audience for consumer "AI". How much do the great majority care if a poem is made up? They want something they can pull out in a friends group. Google et al give them what brings them back. How many pithy aphorisms from Einstein and Chief Seattle were floating around before consumer AI? 🙂
@scottbrim @carrideen I feel the major paying AI consumer has one interest, accuracy of results. People will not be willing, except for zealots of any flavor, to pay for lies or fabrications.
@Jon_Kramer @carrideen Why do you think many of them are inclined to make stuff up?
@scottbrim @carrideen I don't know. I am unfamiliar with the logic that created them. But I highly doubt it is a feature designed in by the programmers.
@scottbrim @Jon_Kramer @carrideen If the output of the LLM is based on statistics: if one text contains the right answer and three texts contain the same wrong answer, it seems likely it’ll go with the wrong answer.
@helianthropy @scottbrim @carrideen I don't think it is that simple. That would be fine for an "sources say that..." answer, but would not be the case for "According to Babbitt v Harris 1976, Michigan civil court..." where there was no Babbitt case... Where the case was fabricated from no sources at all.
@carrideen once upon a time google search worked very well, now it returns hot trash and prompts you for feedback to train the algorithm. Been removed from all my browsers. Just absolutely nuts.

@carrideen

I've pretty much come to the point that I want a printed book with a publication date of 2000 or earlier. Of course the libraries have either dumped books wholesale or made them inaccessible by moving them off-site and making them request-only, because "everything is on-line now".

Ooops.

@carrideen On a lark, a while back, I tried to use an AI to help with lesson planning. It gave me a list of 10 poems to use as companions to Fahrenheit 451. 6 of the 10 poems don't exist and 2 of the last 4 were not theme-based at all; I have no idea why it offered up those - listed on the same page somewhere maybe? And the thing is, I can spot this, but my students can't. I don't even know how to feel about this. All I can really do is caution my students about it, and gently "call them out" on AI content when I spot it. And by that I mean, query them verbally to ascertain if they wrote it, and if they didn't, explain our department's position and have them re-write those parts. It's a tool some people will use, and if they are, they need to do so appropriately.

@carrideen Everytime I read something like this I feel bad for teachers. Even as an undergrad student I’m seeing my peers over rely on AI models and immediately trust the data it spits out with no fact-checking or verification - then being baffled when they get a poor grade because it was wrong or incomplete.

I forget how to trigger it (I don’t daily drive Google anymore) but there’s a -ai tag or something of that sort to disable Gemini results. Might be beneficial to make a mention of that and point out errors like this to the class. People won’t realize that GenAI won’t be helpful to them like they think until it is explicitly mentioned and displayed.

@carrideen We’ve just spent a few decades teaching students that you should trust backed up information (= stuff you verify with internet search) above a single person’s opinion, as your classmate or neighbor can be wrong.

And now when teachers say ”that’s completely false” about something a kid read from a sentence generator, it can rightfully seem like the teachers are the ones mistaken. There is the valid-seeming search result, after all.

@kaisla @carrideen This is extremely bleak, but sadly also extremely true.
@carrideen we’re in for a wild ride the coming decade. What’s real what’s not and does the distinction mean anything to the next generation 😶‍🌫️
Misinformation isn't new. Hopefully, they learn to take the information and not just blindly accept it. That is the level of critical thinking so many people in the current workforce lack. Removing AI doesn't fix that.

@carrideen
Use offline Lists like Copy and Paste with Computers and Libraries. Still easy known Math, databases and copied shared knowledge. A. I. can help too but need secured and watched results. Back to wrote Information loop...

Important is the consent about value of information.

@carrideen When students say their teachers ban Wikipedia as a source of information because it's edited by mere humans, not scholars...

Well...at least it has humans.

@carrideen Thank God libraries still exist. I know that doesn't fix that AI is but it's a hedge against growing AI-powered misinformation and misdirection.
@carrideen General use of AI is dangerous and should be banned. It's making users braindead. It should only be used in the field of science to process masses of data in the hope of finding a needle in a hay stack.

@carrideen slop, it is overwhelming with slop.

And worst of all, this is not by accident.

@carrideen
Are other search engines more reliable than Google? Less AI for example. If they are, maybe telling students not use Google might help. I am using DuckDuckGo now
@carrideen There’s nothing like books, is there?
@carrideen something you may want to tell your students about, adding “-ai” to the search query will give you google results without the AI. I mean, trash LLM generated stuff that’s been indexed will probably still show up but it’s at least a little better