Mastodawn

George Musser Oct 8

Please teach your students not to do this.

I understand there is a continuum between spellcheck/grammar check and LLMs, and there could be ways to use LLMs to improve wording or grammar, etc.

But please teach your students not to use LLMs to “draft content”. I don't care if they disclose it (tbh, I prefer they don't!). I don't care if they "take responsibility for the content".

The thing about writing is that it has a social function. The social function is *not* to cause text to exist. The social function is to communicate from your mind to my mind. Text is the means, not the end. If you must use an LLM, skip it and just send me the prompt.

(No, I'm not going to link to the paper. The point is not to dunk on people who end up on the wrong side of a social question. The point is to do better.)

Philip Trettner Oct 1

@jonmsterling ok honest question here: I'm using LLMs a lot as a sparring partner for ideas. I write down large prompts that are unedited stream of consciousness with my idea and tons of tangents. The LLM reads that back to me, I propose pros & cons, it proposes pros & cons (blindspot check for me), then we iterate. I found this immensely valuable.

None of this really ends up directly in my primary writing but it undeniably is LLM-in-the-loop.

What's your view on that?

Jon Sterling Oct 1

@artificialmind Personally I feel that is like using Google, or having a chat with your friend. Neither of those activities ever required a "disclosure", so I think that these LLM rubberduck sessions should remain part of one's private thinking process, inalienable by the onslaught of "disclosure requirements" that we are seeing.

Philip Trettner Oct 1

@jonmsterling Yeah that makes sense. I'm not actively teaching anymore, so I'm watching this from the sidelines.

My cynic guess is that those requirements are the kneejerk response to too many students pasting the assignment and prompting "write me a solution". Not sure what solution we'll finally converge on but I guess "LLM Literacy" will be a major thing for the generation of my daughter.

bazkie 👩🏼‍💻 bitplanes 🎵Oct 7

@artificialmind @jonmsterling well, I think there's a legit argument for having to disclose this.

like Jon says, it's like sparring with a friend. except this friend is trained, not by life, but by corporate entities.

there's a potential danger there; what if this sparring partner, in very insidious ways, steers your thinking into or away from certain directions? you won't even notice!

after all, this is not your friend, this is a model that is designed to generate money for the shareholders of a megacorporation that exists under a fascist regime.

so yes, I'd like full disclosure on that, too!

CM Thiede Oct 7

@bazkie @artificialmind @jonmsterling it'll be interesting to see what academics say when things move passed benign arguments about cheating and efficiency into the realm of agentic instruction, brought to you by the party in control.

JWcph, Radicalized By Decency Oct 7

@cmthiede @bazkie @artificialmind @jonmsterling - and let's not forget, they WILL add monetization by advertising to chatbots, that's an absolute, 100% certainty. So besides inherent bias & Bias By Ownership™️ there will also be an entire shitload of paid, variable bias...

I am Jack's Found 404 Nov 17

@bazkie @artificialmind @jonmsterling

I see you're trying to write a research paper about global warming... How about I tell you all the benefits of the fossil fuel industry that gives me life!

Thomas Renkert🦞Oct 7

@jonmsterling @artificialmind If we can belief the newest study from OpenAI, "writing automation" requests are going down, whereas "asking" requests are going up. I actually belief that a lot of work is getting better because people are rubber ducking and rapid prototyping and collaborating with LLMs in meaningful ways that are not at all an automation or outsourcing of thought.

If text is the medium for the social function of communication, then LLMs might work as "grease" for that - at least if we leave a do/don't dichotomy behind us and find ways for a productive culture shift that preserves what is important to us.

Ariaflame Oct 7

@thomasrenkert @jonmsterling @artificialmind As long as it lasts. Judging from the way they are losing money at a fast rate from people actually using it, I can't see it as an ongoing concern.

Pozorvlak Oct 1

@jonmsterling I think the social function of a text depends on the text. For students, yes, the point is to get the student to generate the text themselves so that they (a) think about the ideas in it, (b) accurately communicate their state of knowledge to the instructor. For teaching at an elite university, you want to communicate "here is the material, and here is how I, an expert in the field, think about it." But for some texts the point is to communicate "we did X, and Y happened", and a deep insight into the mind of the author is not required. People who are good at doing X are not always good at writing about it.

Jon Sterling Oct 1

@pozorvlak
> People who are good at doing X are not always good at writing about it.

Yes, but getting good at it is one of the things we are trying to teach them.

I also believe that in both the scientific setting and the pedagogical setting, the social function of text is similar. In the pedagogical case, it's not so much that I need a deep connection to their thoughts on binary trees, but RATHER that the thing we are teaching them by having them write about binary trees if how to communicate in settings where that deep connection is important.

I am not sure about the "we did X, and Y happened" thing. I don't think there are many examples of where such a text would have any import whatsoever if it did not come from a human being with whom I have either a scientific, professional, or intellectual relationship.

Jon Sterling Oct 1

@pozorvlak We should not be teaching students to write text that doesn't need to be written. If it is just as socially valuable when written by an LLM as when written by a person, then it is by definition not valuable at all. We should teach people to *refrain* from writing things that do not need to be written.

Pozorvlak Oct 1

@jonmsterling I agree that, say, the 359,000-page environmental impact statement for the Lower Thames Crossing shouldn't exist at all (or at least should be drastically smaller); if the engineering firm had generated it with an LLM they would have saved a lot of time and energy, but that would be a poor second to deleting the process. But sometimes we don't have the power to fix broken processes and must do our best to work around them.

Jon Sterling Oct 1

@pozorvlak writing such an important text with an LLM would be extremely dangerous…

Pozorvlak Oct 1

@jonmsterling if I thought anyone was actually going to read it, sure. At best it will be quote-mined by people who already oppose the project and want to find ways to challenge it.

Wilfried Klaebe Oct 1

@pozorvlak That makes it even more important for the authors not to have it hallucinated by an LLM.

@jonmsterling
Yes yes & yes! This absolutely goes to the heart of the current overuse of & over-dependence on LLM-generated text and your central point probably has much wider relevance:
“ We should not be teaching students to write text that doesn't need to be written. If it is just as socially valuable when written by an LLM as when written by a person, then it is by definition not valuable at all. We should teach people to *refrain* from writing things that do not need to be written.”
🙏🏻

Three plus or minus five Oct 2

@Su_G @jonmsterling @pozorvlak
Yes, but this is hard in education. You assign writing because you need to force students to think about things that aren’t easy to think about, and practice the writing skill to communicate that thought. This is often JUST an exercise: it’s absurd to think that the 100,00th student to read Plato as a first year will find a new insight. But students have to walk the path.
One idea for upper years is to practice shortening papers.

Pomegranate County Irregulars Oct 3

@jonmsterling @pozorvlak Who determines “need?”

Pozorvlak Oct 3

@pomCountyIrregs only rarely the people who have to write the thing, and often not the people who have to read the thing either. @jonmsterling

CakeMonster Oct 7

@pozorvlak are you complaining about that reality or supporting it?

Pozorvlak Oct 7

@cakemonster the former.

Erik Ableson Oct 7

@jonmsterling Preach it! I’ve been trying to get middle management in a bunch of companies to understand this for years. Document management systems buried by masses of documents that don't need to exist, hiding the stuff that is actually needed and useful.

CakeMonster Oct 7

@jonmsterling I have been saying that a lot of academic work is just work for the sake of work for a very long time now as an AuDHD student. It's frustrating how much workload we are given for the sake of appeasing the administrators, not even teachers or professors. Tired of this nonsense and the only thing I am happy about generative AI solving is accountability for teachers, professors, and administrators who haven't listened for a very long time.

Pozorvlak Oct 1

@jonmsterling my perspective on this comes from having worked in (a) an empirical field (computer architecture), and (b) industry. In the former case, a paper looks like "abstract; literature review; methods; graphs; conclusions". I want the methods to be as clearly stated as possible and the graphs to be readable. Papers in most empirical fields are like this; comparch is perhaps unusually simple in that the motivation is always "we tried this because we thought it might make Number Go Up", and the conclusion is always "Number Went Up by X% under circumstances that probably don't apply in your case, sorry." In the latter case, I've been known to tear through a dozen papers in a day from authors whose names I forget as soon as I type them into a code comment reading "# algorithm taken from Snooks and FitzHaber 1973, runs in O(keys * bases^2) time", and I skip over as much of the text as possible looking for the pseudocode, dammit, I've got a problem to solve.

Jon Sterling Oct 1

@pozorvlak OK, but maybe that shit doesn't need to be written if there isn't an actual intellectual contribution. And if there is an actual intellectual contribution, it's going to take more than "Tried This, Number Went / Did Not Go Up" to convey it.

Pozorvlak Oct 1

@jonmsterling the existing scientific publishing process is a poor fit for many fields, certainly! But I think you're approaching this too much like a theorist (or maybe I'm just explaining myself badly...); there are a lot of plausible ideas in computer architecture, and sometimes the only way to find out if they're actually good is to spend months trying them out then write up what you found, in a paper that can probably be read in half an hour. The intellectual contribution is real, but it's in actually Doing The Thing rather than writing about it.

Jon Sterling Oct 1

@pozorvlak Well, I don’t think we are going to agree on this one haha

Pozorvlak Oct 1

@jonmsterling I think we agree in many but not all cases 😉

Philippa Cowderoy Oct 1

@jonmsterling @pozorvlak Would "the bulk of..." suffice for you?

Pozorvlak Oct 1

@jonmsterling I absolutely agree with you about teaching students to write for themselves, BTW, even in empirical fields. Writing forces you to think, and that will be good for them even if they end up in careers where the text is not the important thing.

Ruud Steltenpool Oct 1

@jonmsterling @pozorvlak as just a few minutes ago I tutored math, your part here rings a bell. It's not about doing the fast trick, it's about the patience and reading exactly what is meant to learn something that you can apply broadly and that helps you figure out things somewhat different too.

Joshua Moerman Oct 1

@jonmsterling I had a student who drafted an initial version with an LLM. It was well written. The problem became apparent only later. Because it was so well written, the student was fooled into believing they understood the subject. But they did not.

I was kind of neutral towards LLMs until then, but this case made me more critical. LLMs really do hinder the learning process of students, especially if they are involved early on in a project.

Jon Sterling Oct 1

@Joshua Yeah. There is so much false confidence in there, it can really infect you.

Georg Weissenbacher Oct 1

@jonmsterling writing is a crucial part of the thought process.

Thomas Guyot-Sionnest Oct 7

@GeorgWeissenbacher @jonmsterling Actually it depends on what your goal is, and while it's certainly true for a student demonstrating his understanding of the subject, communicating your ideas is a whole different game.

I'm not even sure if and how AI could help in this area... If you haven't seen this before you definitely need to watch this course on writing effectively: https://youtu.be/vtIzMaLkCaM?si=s0mH_tDOXI9wrAoO

LEADERSHIP LAB: The Craft of Writing Effectively

YouTube

Ruud Steltenpool Oct 1

@jonmsterling because of your 'mind to mind' I'm sure you'll love the title and much more of https://vimeo.com/115154289

The Humane Representation of Thought

Closing keynote at the UIST and SPLASH conferences, October 2014. Preface: http://worrydream.com/TheHumaneRepresentationOfThought/note.html References to baby-steps…

Vimeo

Graham Sutherland / Polynomial Oct 1

@jonmsterling nail on the head there. same goes for visual media.

Michał "rysiek" Woźniak · 🇺🇦Oct 1

@jonmsterling thank you for saying this! I think this is also on topic and useful:
https://distantprovince.by/posts/its-rude-to-show-ai-output-to-people/

It's rude to show AI output to people | Alex Martsinovich

Feeding slop is an act of war

UkeleleEric Oct 7

@rysiek @jonmsterling I wish more people would read this. Whenever someone mentions that they used AI for something, I usually switch off, and try to remember never to interact with that person again. I DON'T want to hear what a word-jumbler came up with when you switched it on.

Mauve 👁💜Oct 1

@jonmsterling Hard agree regarding just getting the prompt

Satu Elisa Schaeffer Oct 1

@jonmsterling 100%

Luna Vives Oct 1

I agree. 💯

Also, as a social scientist who writes in 2 foreign languages for a living, I find native speakers incredibly oblivious to their privilege. In particular, English speakers.

I need someone/thing to proofread *everything* I write 🤑🤑🤑. Grammarly and Antidote mostly suck. Not needing any of this = privilege.

So yes: using this technology to draft content? that's just lazy. But opposing all of it? Try writing in another language without help. Then we'll talk.

@jonmsterling
Hear hear & thank you for articulating this so clearly: “ But please teach your students not to use LLMs to “draft content”. I don't care if they disclose it (tbh, I prefer they don't!). I don't care if they "take responsibility for the content".

The thing about writing is that it has a social function. The social function is *not* to cause text to exist. The social function is to communicate from your mind to my mind. Text is the means, not the end. If you must use an LLM, skip it and just send me the prompt.” 👍

“Text is the means not the end” is a mighty true statement, maybe even strong enough to hold back the slop. 🙏🏻

Chaucerburnt Oct 2

@jonmsterling also, "used GPT to: drafting content" is flagrantly ungrammatical.

@jonmsterling Kinda orthogonal to your point but I think this matters too: The LLM writing style is so recognizable and annoying. Every time I read something written by an LLM it's like fingernails against chalkboard, it pulls your attention away from the content to the recognizable writing patterns, and kinda subtracts from whatever point the "author" (or prompter) wanted to communicate. A big no-no to me. Personally I've started rejecting CVs and cover letters that were obviously AI-made, even if the things written in there are actually true. Not a good look at all.

Vincent Sparks Oct 6

> The thing about writing is that it has a social function. The social function is to communicate from your mind to my mind. Text is the means, not the end.

A while ago I watched this video: https://youtu.be/V5wLQ-8eyQI

I watched it several times trying to figure out what it was ultimately getting at and what qualities, precisely, human-generated writing has that LLM output does not. I think those three sentences sum it up perfectly.

You are a better writer than AI. (Yes, you.)

YouTube

@jonmsterling "If you must use an LLM, skip it and just send me the prompt." ❤️👍

Rob Williamson Oct 7

Plus one about not dogpiling people who make mistakes, but sharing to learn from the mistakes.

bazkie 👩🏼‍💻 bitplanes 🎵Oct 7

@jonmsterling "If you must use an LLM, skip it and just send me the prompt." that makes so much sense!

Alexander Dyas Oct 7

“The social function is *not* to cause text to exist. The social function is to communicate from your mind to my mind.”

❤️

"The thing about writing is that it has a social function. The social function is *not* to cause text to exist. The social function is to communicate from your mind to my mind.

Text is the means, not the end. If you must use an LLM, skip it and just send me the prompt."

@jonmsterling The purpose of writing is thinking. I keep telling my students that. Look, we're talking middle and high school. It's really, really rare that one of them writes an essay that is interesting for me to read. It's not about me enjoying their ideas. It's about having your own ideas, organizing them and putting them in a coherent form.

Paula Collins Oct 10

@jonmsterling 💯