Matt Blaze
One student's paper included footnotes for each "fact that ChatGPT got wrong" that was relevant. Interesting flex!
May 12 at 2:10amWeb
Show threadMatt BlazeMay 12

What the student did was write the paper, and then asked ChatGPT for comments on it. And, of course, some of the comments were wrong.

Which the student caught.

This might be the most useful way to incorporate genAI into courses I've seen. No my idea - the student did this on their own!

Show threadtlarivMay 12
@mattblaze
Brilliant!
Show threadAdam Shostack  May 12
@mattblaze There’s some sample assignments out there with a “let ChatGPT write the paper, then critique it” with the LG being to see how mistakes get introduced. Pedagogically interesting if you have time in syllabus
Show threadMatt BlazeMay 12
@adamshostack I like the idea of starting with the student's work and then document essentially a round of comments, some of which will likely be helpful and others less so.
Show threadMatt BlazeMay 12
@adamshostack It's essentially having the student deal with reviewer #2.
Show threadAdam Shostack  May 12
@mattblaze I think the key is what’s the learning goal? Both can be useful. I’m really concerned with llms as consumate bullshitters , but if you’re training grad students to deal with program committees, then r#2 is important.
Show threadMatt BlazeMay 12
@adamshostack I think being able to engage with unreliable feedback is extremely useful (and demonstrates a very high level of knowledge).
Show threadhalfaMay 12
@mattblaze @adamshostack this, 💯
Show threadQuinn NortonMay 12
@mattblaze @adamshostack I am extremely amused at the thought of LLMs taking the role of bitchy pain in the ass, denying those jobs to human operators?
Show threadJo - pièce de résistanceMay 14
@mattblaze I agree. 100%.
Show threadClaudius LinkMay 12
@adamshostack @mattblaze
I fear that if your start with the LLM work you risk getting some kind of anchoring effect.
You are less likely to discover different pathways not taken by the LLM.
And I would also expect less LLM errors on the path reducing the learning
Show threadNoodlemazMay 12
@mattblaze sounds like mostly a waste of time though?
What would be lost if they didn't do that?
Show threadGeorge BMay 12

@noodlemaz @mattblaze

They're students. Giving them an intuitive sense of how inaccurate LLMs are is a lesson worth teaching even if they get nothing else out of it

Show threadNoodlemazMay 12
@gbargoud @mattblaze do you think homework is the only way they'd encounter this?
Show threadNorman WilsonMay 12
@noodlemaz @gbargoud @mattblaze Do you think homework is a good exercise to help them learn? If not, what is the point of homework?
Show threadNoodlemazMay 12
@oclsc @gbargoud @mattblaze well I was going to share something about the known downsides or even harms of homework but guess what every piece I click on seems to be AI rubbish
Ew
#noToAI #genAIslop
Show threadMatt BlazeMay 12
@noodlemaz I disagree. I think being able to distinguish (and defend one’s writing against) unreliable feedback while also potentially absorbing useful feedback is extremely useful, and develops and demonstrates a very high level of understanding.
Show threadNoodlemazMay 12
@mattblaze fair enough, whatever makes teachers' and students' lives easier in this hellscape tbh
Show threadMatt BlazeMay 12
@noodlemaz It’s not about “making life easier for teachers”. It’s about finding ways to help people learn. Maybe this is. Maybe not.
Show threadNoodlemazMay 12
@mattblaze implicit in that was meant to be 'easier to do their jobs' i.e. Helping people to learn.
Show threadDavidM_yegMay 12

@mattblaze

While academia can be full of all kinds of unnecessary obstacles, many of the processes of academic science that seem the most arduous were developed precisely to manage and weed out both unreliable content and feedback. Bullshit isn’t new, what is notable about “AI” is the sheer volume and apparent ease of use

@noodlemaz

Show threadTomMay 12

@mattblaze Kinda like what you have to do on this thread! 😅

I think what the student did was very clever, not only showing mastery of the subject, but also finding improvements in a flawed system (LLM use.)

Show threadKateTMay 12
@mattblaze a great outcome but only possible if the student has enough domain knowledge to spot the errors.
Show threadJo - pièce de résistanceMay 14
@KateT @mattblaze
Only if they actually learned what they needed to learn, and not just regurgitate information without comprehension, without digesting it.
Show threadaprilfolliesMay 12
@mattblaze I would absolutely award points for that.
Show threadCapt. TonyMay 12

@mattblaze kudos to that student for fact checking the ai slop, but just do the research and write the paper without it and show off your understanding of the subject along with appropriate critical thinking and writing skills

*edited for spelling errors

Show threadMatt BlazeMay 12
@mermaidchaser The student did fine.
Show threadDarryl RammMay 12

@mattblaze

...professor quietly checks other students papers for these mistakes.

That could be quite a power-flex by the student if they thought others were relying on ChatGPT.

So maybe points for that as well 🙂

Show threadlongpantsMay 12

@mattblaze

The new generations are blazing the way on how to deal with these tools, I weirdly trust them that it'll turn alright somehow.