Inky says "What the hell?!"17h ago
myrmepropagandist

RE: https://mastodon.social/@arstechnica/116397070356098122

In an ideal world courses would have pass/fail grades if they have grades at all. Grades get in the way of learning. (Students turn to LLMs to save their grades.)

So, I think part of this tension goes back to what we grade and why.

I'm lucky that in math I only ever assign grades for in-class tests.

In my CS classes I lean a lot on sharing student work. Fortunately my students like to show off for each other. They are excited to see what I and their peers think of their projects.

Apr 13 at 11:35amWeb
Show threadmyrmepropagandist1d ago

Teacher feedback is precious. It takes time to read and write comments and provide guidance. Doing that kind of work on LLM slop is depressing. The teacher could have been writing feedback for a student who did the work and might learn from it.

How can the way we present this feedback emphisize its value?

What about "you may submit 3 essays per term for feedback" the feedback isn't a grade, it's just comments to help you when you write the final essay in class without an LLM?

Show threadmyrmepropagandist1d ago
When I was in school I always wished my teachers would write more comments on my essays. I just wanted someone to read them and be impressed. I don't think that's changed.
Show threadmyrmepropagandist1d ago

Feedback is also a limited resource and students don't understand this since they think teachers have infinite time.

Maybe the way that it's presented needs to make it more clear that this is something of value, not a punishment... which is how younger kids can see it.

eg. "Decide what you want me to read, I only have time to do this two times."

I think fewer students would give LLM content in that context.

It is, frankly, rude to give someone a bunch of LLM text to read.

Show threadmyrmepropagandist1d ago

I had a 6th grade student who I asked to design a pattern to print on the laser cutter. She was excited to get feedback and use the laser cutter.

Her pattern was rushed and ... just not very good. So, I asked her to re-draft it with more care. This was a big ask. In 6th grade that's a new skill.

"Why didn't you write more feedback?"

"I'm not spending more time on feedback than you did drafting. Slow down."

She got it eventually, but there is more work to be done there.

Show threadMoon joy1d ago

@futurebird I can totally see the 6th-grader's perspective - I too was always impatient to learn how to do everything perfectly! (I'm so glad there weren't LLM chatbots when I was a kid, I'd have been susceptible to their siren song for this very reason!)

But of course now I see yours so much more clearly - learning to be careful is so much more useful than whatever the outcome of that particular laser-cut could be!

Show threadJencel Panic1d ago
@futurebird true, unfortunately our whole society works with grades and punishments. Even at work we are asked to grade our colleagues
Show threadPatrick Lam 1d ago
@abuseofnotation @futurebird pass/fail as mentioned elsewhere in this thread is something that I believe in as well. Unfortunately I have to give more detailed grades than that.
Show threadmyrmepropagandist1d ago

@va2lam @abuseofnotation

I'm. in the same boat. Either the kid learned algebra and will be fine next year learning more math... or not. And the later is rare.

It feels kind of unfair to send them all along with different grades that end up being used for all kinds of things that have nothing to do with what courses they need to take.

Show threadDon Ray1d ago

@futurebird

The Robert Frost quip (from his professor days) comes to mind:

“I am no perfunctory reader of perfunctory papers.”

Show threadcpkimber1d ago
@futurebird I only have about 20 or 30 students per semester, but I have all the time in the world for the ones that give me their time and truly try to understand what I am teaching.
Show threadAsakiyume1d ago

@futurebird

--to the "something of value, not a punishment" part,

I was like you in liking feedback, but I think it's because I received mainly positive or encouraging comments, which I think means I had teachers who valued that. I think that some kids have the experience of teachers who just write. "No" and "Explain" and "Word choice??" which can be deflating and discouraging and can lead them to not want feedback.

(And/But it's also true that they think teachers have unlimited time!)

Show threadLizzie1d ago
@asakiyume @futurebird

Slightly tangential but I fairly recently learned that people with ADHD tend to be oversensitive to negatives due to difficulty regulating emotions, and even the most well meaning feedback or critique can feel about the same as being completely torn down. So unfortunately for some people even constructive and encouraging feedback can ring out negatively.

Hell, I've seen it with myself lots, and only now know why.
Show threadAsakiyume1d ago

@AngelicAura @futurebird

Yeah, that's another thing a teacher can learn--the sensitivities of individual students. But it's hard, given how overburdened they are.

Show threadLizzie1d ago
@asakiyume @futurebird
That is true, I would never expect a teacher to go to so much additional effort. Well, at least not with how education is set up in much of the world nowadays.

Just wanted to offer another perspective!
Show threadAsakiyume1d ago
@AngelicAura @futurebird Very glad to have it! There are so many sides and aspects to any idea, discussion--anything really.
Show threadsecretsloth1d ago
@futurebird it really sounds like using an LLM would be the fastest way to suck joy out of pretty much anything. I'm glad I'm not in school now, since I'm autistic and apparently our written work tends to get flagged as LLM written pretty often, at least I don't have to deal with having to fight that accusation. I wonder though whether folks who initially cheat using LLMs, eventually stop? Probably not all, but I'd bet some do. 🤔 I wonder how common it is.
Show threadIris Young (he/they/she) (PhD)1d ago
@futurebird this would have helped me a lot (and also frustrated me to no end, honestly). I did not have that level of empathy developed yet, to my perpetual shame.
Show threadAna Bollocks / Kristin C.1d ago

@iris @futurebird I didn't either. Maybe it's a development age thing, at least to an extent?

Thinking out loud... maybe spelling out the purpose of the feedback would help, especially if the essays are voluntary. And that it takes time for you to give might also help. I know it seems obvious but... kids (and adults!) are where they are. The comment about not spending more time on feedback than was spent on the task is perfect, but maybe more kids need to hear it.

Show threadAna Bollocks / Kristin C.1d ago
@iris @futurebird Also, granted I was a kid who didn't worry about grades in the first place but - I'm not totally convinced kids use LLMs only to save grades. They want to spend time on other things too. I *liked* learning and even homework. But that homework was perpetually done during the class before it was due, bc I had other things I wanted to do after school. Or I just forgot about it until it was the next day already.
Show threadBruce Simpson, Ph.D.1d ago
@futurebird Such strange concepts from a GPT. Honesty, understanding. These concepts do not exist in the world of the LLM except in terms of model weights, and I would rather we had a less contrived human-computer interface here, where we meet in the middle, rather than pretending it is something that it is not.
Show threadTael 🔜 AC261d ago

@futurebird "It is, frankly, rude to give someone a bunch of LLM text to read."

100%! Extremely disrespectful to give advice on something and they respond with the ChatGPT output for their question.

Show threadPaul Chernoff1d ago

@futurebird Flashbacks to a college course. Supposedly we were supposed to be learning critical thinking but the goal was to have students adopt political opinions.

If was a film course and we were to write in a journal after seeing each movie. We would see only comments from the graders (grad students). We were graded but didn't get to see the grades, only comments.

When we finally saw the grades they were much lower than most people expected. So the quality of comments weren't that good.

Show threadHugo Mills1d ago
@futurebird It once took me about ten minutes to work out that my English teacher had written "you must learn to write more neatly" at the bottom of an essay...
Show threadTak!1d ago
@futurebird I keep seeing teachers complain about getting slop essays, and I keep wondering why teachers keep assigning essays (for grading, as you say) in the Slop Era
Show threadCharles ☭  is a Green1d ago

@Tak @futurebird

Redesigning all of a course's assessments in a way that meets the learning outcomes and is fair to people who have accessibility needs is a non-trivial task.

In some cases, it needs committee sign off. Or maybe the university as a whole is deciding what they're going to do differently and you need to follow the institution-wide guidelines.

There's actually a lot of care put into assessment. There's a QA process.

An effective response to slop also requires tackling the problem of slop research, so it's also more complicated than just moving to proctored exams.

Student motivation is also super tied to marks and, at least at the university level, those marks and their percentage value for the course is how students decide how to prioritise their time. Some scholarships also take marks into account.

Rethinking the entire assessment model is necessary, but the resulting changes need to meet the needs of students, the institution, and, alas, funders.

Show threadTak!1d ago
@celesteh @futurebird that's fair, my impression of my own secondary and university education was that there wasn't really any substantive oversight or QA - I didn't consider that many educators could be handcuffed by organizational requirements
Show threadKalenXI 🏳️‍🌈1d ago

@futurebird For me going through school (at least pre-college) I never really valued teacher feedback. Because most of the feedback was not "what does the teacher think of my work?" it was "how closely did I follow the rubric?"

It felt like the actual content of my work was largely irrelevant since we were graded entirely on how we wrote not what we wrote.

And every time I brought this up to a teacher the response I got was "Well we have to teach to the curriculum."

Show threadKalenXI 🏳️‍🌈1d ago
@futurebird I imagine an LLM would probably have gotten better grades than me since strictly following a given format is something they're pretty good at.
Show threadSteve Lindsay1d ago
@futurebird When I was in grade 6 I worked super hard on a school paper for the first time. It was about skin myelinization and the title was Man the Rainbow. My teacher Mr. Slade picked up on my investment in the paper and asked me to stay after school so he could read it and give me feedback that very day. He gave me nice comments and an A+. That had a big and lasting impact.
Show threadGeorge B1d ago

@futurebird

(Speaking as someone who is not a teacher and has not been a student for many years so grain of salt yadda yadda)

The problem with LLMs at school seems very similar to the problem with LLMs in a lot of other scenarios: people added a bunch of bullshit bureaucracy to try standardizing non standard things at scale and rather than admitting it was a failure and ripping it out, they're now just attaching it to the bullshit machine and calling it an efficiency win.

Show threadAndy Lee1d ago
@futurebird "They are excited to see what I and their peers think of their projects." ❤️
Show threadThree plus or minus five1d ago

@futurebird
Grades are a useful lever but also a crutch. There are two problems the address in my mind: students who want the degree but don’t want to learn, and student who want to learn but don’t believe that the work will help them.

Both are easily moved with a little incentive. But the era of the LLM suddenly changes the cheating cost to reward ratio. I’m glad I’m not teaching anymore…

Show threadNindelofocho1d ago
@futurebird As someone who teaches I agree with this big time. In language learning a student may make a bunch of small errors but still demonstrate a grasp of the language at the level but would otherwise fail if grading from a rubrick. I feel like this holds students back at best and at least demoralizes students.
Show threadMusta dawned on me thusly1d ago
@futurebird teachers are the front line working against #idiocracy: https://newsie.social/@blackburied/116395486572016457
Musta dawned on me thusly (@[email protected])

#AI isn’t making people stupid; just making an old problem easier to ignore We’ve always outsourced thinking; let someone else handle everything, and your own ability fades. Like dealing with a control freak: they take over, and you forget how to steer AI just does this faster and more quietly The real issue isn’t AI—it’s whether you let something replace your thinking or build it Teachers make the difference: they make you think Everything else—AI included—you make go either way

Newsie
Show threadThe Worst Timeline1d ago

@futurebird one of my favorite/best-remembered exams was when the prof took some code we wrote at the start of the semester, broke it, and had us fix it. It was extra work on his part because each exam ended up with bespoke elements, but it was clearly effective if I remember it ~35y later.

Depending on how well you wrote (and commented!) it was either an easy or horrible task.

With genAI-sourced code—particularly if “it works the first time—there are lots of opportunities to trap students who don’t really know what the code is doing with that sort of exam challenge.

Show threadKaare Mikkelsen1d ago
@futurebird i absolutely agree with the 'grades get in the way of learning', at least for the weak students. Saying it out loud (which I do, to both students and colleagues) makes me sound kind of extreme though...
Show threadElizabeth1d ago
@futurebird I taught high school science in a system with a state-mandated standardized test at the end which determined graduation but did not affect the students' grade in class. With 90-plus students and 70 class days to cover astronomy, geology, meteorogy, and oceanography, it was difficult to give individual feedback unless the student was willing to stay after school - even more difficult in a working-class suburban school where most sudents' only transportation home was the bus.