Debby Cotton
Edutooters! I’d be really interested to hear any thoughts on this paper on #ChatGPT What has been your experience with students and the new AI possibilities? Have you found uses for it in academia?
Chatting and Cheating: Ensuring academic integrity in the era of ChatGPT'
https://edarxiv.org/mrz8h/ @edutooters #edutooter
Jan 13, 2023 at 12:54pmWeb
Show threadKathryn WJan 13, 2023
@ProfDCotton Don't have anything to contribute but just wanted to say thanks for sharing the paper, it's a really useful overview of the benefits/ concerns
Show threadDebby CottonJan 13, 2023
@kathryn Thank you. It was fun to work on!
Show threadDan HattonJan 15, 2023
@kathryn @ProfDCotton I took a look at the paper. Nice work. My two penn'orth: in the ChatGPT-written first part, many of the prompts given to ChatGPT are ambiguous as to taxonomic level, e.g. 'How can academics prevent students plagiarising using ChatGPT' can be interpreted as 'list the ways academics can prevent students plagiarising using ChatGPT' or as 'evaluate the likely effectiveness of each of the ways academics can...' ChatGPT has attempted only the former.
Show threadDebby CottonJan 15, 2023
@only_ohm @kathryn Yes, good point. We did try to refine the questions but I think we could have done more to prompt more detailed answers in places - it is quite dependent (obviously) on what you ask it to do… Having looked at a lot of output now, it feels quite ‘samey’ too - I kept wanting to edit but resisted!
Show threadDan HattonJan 15, 2023

@ProfDCotton

@only_ohm @kathryn

It wasn't a criticism of your methodology - I think it's interesting to give the AI the ambiguous prompt and find out that it "chooses" to answer at the lower taxonomic level.

Show threadBob IrvingJan 13, 2023
@ProfDCotton @edutooters Just read this from Alfie Kohn this morning, and I think he's correct: https://twitter.com/alfiekohn/status/1613885037723762694?s=20&t=wEb2jVBx8cPMCCULmKSMXQ
Some of the issue may be that assessments are not necessarily measuring authentic learning.
Alfie Kohn on Twitter

“1/3 The real problem, I've argued, isn't that students cheat but that they're assessed in such a way that cheating is possible (because of an emphasis on memorization). Similarly, it's not that some kids wangle extra time (https://t.co/pwPZPZt3m3) but that tests are timed...”

Twitter
Show threadDebby CottonJan 13, 2023
@birv2 @edutooters Yes, absolutely! Maybe this will be a wake up call for educators. But I fear that many will use it as a retreat to even more conservative assessments (F2F exams)
Show threadBob IrvingJan 13, 2023
@ProfDCotton @edutooters What do you mean by F2F exams? Pencil and paper in the room with the prof? Eww.
How about we shift towards intrinsic motivation, student choice, and personalized learning? Lots of big fancy ideas there, but I agree with you. This might be our moment to advocate for learning like that!
Show threadDebby CottonJan 13, 2023
@birv2 @edutooters Yes, exactly. There’s what we should be going (authentic assessments etc) and what is more likely to happen (more exams) alas…
Show threadBob IrvingJan 13, 2023

@ProfDCotton @edutooters The two major barriers to any substantive change:
1. We've always done it this way.
2. We've never done that before.

I don't really see a way to jerry rig the educational system, unfortunately. However, AI just might give us the impetus needed to get all parties' attention. Hopefully, this will be a "sputnik moment".

Show threadZingerLearnsJan 13, 2023
@birv2 @ProfDCotton @edutooters I appreciate this point and hope that we see more of
1) how can we do better and improve
2) how can we help students learn
as pervasive mindsets and approaches, but I certainly see more of the culture you share than this.
Show threadBob IrvingJan 13, 2023
@ZingerLearns @ProfDCotton @edutooters Yes! It should be about putting learning at the center. And by learning, I don't mean the ability to memorize things, though there's a place for that of course (like multiplication tables). Isn't part of the problem the prevalence of standardized testing, which skews towards measuring memorizables?
Show threadZingerLearnsJan 13, 2023
@birv2 @ProfDCotton @edutooters
Yes for sure, evaluation and short term improvement policies perpetuate a system where initiatives are constantly changing, reminding me of the "flavor of the month" problem and one of the reasons teachers become resistant to change (because there is not persistent long lasting efforts that can actually produce change).
Most successful initiatives I have seen take 3-5 years of persistent effort. This is one of the reasons I have moved away from participating in or providing short term PD.
Show threadBob IrvingJan 13, 2023
@ZingerLearns @ProfDCotton @edutooters Yes, I get teacher fatigue with the flavor of the month PD. I think we need more than bandaids to keep going with the present system.
Show threadLew RileyJan 13, 2023
@birv2 @ZingerLearns @ProfDCotton @edutooters Yes! This! It's helpful to put memorization in its place — laying ground work for later application. Memorizing a body of knowledge is necessary in some contexts but not sufficient to becoming "fluent" with material.
Show threadDan HattonJan 13, 2023
@ProfDCotton @birv2 @edutooters
Absolutely, although I will caution that there are tasks in graduate life for which a closed-book, timed exam _is_ an authentic assessment. Think the engineer on the emergency response team dealing with a sudden failure of domestic water supplies to an area; the trial lawyer cross-examining a hostile witness; or the politician being interviewed for a TV or radio show.
Show threadBob IrvingJan 13, 2023
@only_ohm @ProfDCotton @edutooters Totally agree, though I would posit that the examples you cite are authentic applications of learning. Not the same as a timed exam on material neither chosen nor relevant (usually) to a student's own interests. That's my take anyway.
Show threadDebby CottonJan 13, 2023
@birv2 @only_ohm @edutooters Yes. And not that I’m saying never do exams either. Just think it will be a shame if a knee-jerk reaction…
Show threadEd RossJan 13, 2023
@ProfDCotton @edutooters You might like this, on making it *ask* questions: https://www.linkedin.com/pulse/yes-chatgpt-can-answer-exam-questions-write-them-too-justin-shaffer/?trackingId=bqjIA%2FO4SamvliUbK%2FPJAQ%3D%3D
Yes, ChatGPT can answer exam questions. But can it write them too?

There have many stories published since ChatGPT came out last November about the potential of college students using the AI to write essays, answer exam questions, and otherwise skirt the educational honor system. But what about the converse - can we as instructors use ChatGPT to write exam question

Show threadDebby CottonJan 13, 2023
@edaross @edutooters That’s fascinating! I’ve asked it to do quite a lot of different tasks and it’s pretty adaptable. Writing policies etc which I find dull it does very easily!
Show threadEd RossJan 13, 2023

@ProfDCotton

It certainly is.

I'm just getting it to write a version of the game Snake, but instead of a snake growing, it is ants 🐜 joining the back of the line when they eat sugar 🧂 (yes, that's listed as salt, but it'll have to do).
My trouble is the number of times I have to get it to update something means I'm hitting the "Too many requests per hour" limit 🤭

@edutooters

Show threadDebby CottonJan 13, 2023
@edaross @edutooters I’ve no idea what that game is but this made me giggle (especially the salt/ sugar emoji!)
Show threadEd RossJan 13, 2023
@ProfDCotton @edutooters Quick preview in the image. Google has their own version at https://www.google.com/fbx?fbx=snake_arcade if you want to play a working version now
Google

Show threadDebby CottonJan 13, 2023
@edaross @edutooters thanks!
Show threadEd RossJan 13, 2023
@ProfDCotton @edutooters You are welcome. I've now gotten as far with it is I'm going to tonight I think - you can copy the code from https://pastebin.com/raw/6BXc1b0h then paste it into a notepad file, save it as game.html then run it (p.s. generally don't run random code a stranger has sent you!) so you can see it working if you would like