Eric ScoutenFeb 22, 2023
Charlie Stross

Reddit: "My friend is in university and taking a history class. The professor is using ChatGPT to write essays on the history topics and the students need to mark up its essays and point out where ChatGPT is wrong and correct it."

https://old.reddit.com/r/ChatGPT/comments/117gtom/my_friend_is_in_university_and_taking_a_history/

Honestly, this is great! The students learn to check sources, discover that ChatGPT is unreliable, *and* can't use it to generate essay question answers for them, all at the same time!

My friend is in university and taking a history class. The professor is using ChatGPT to write essays on the history topics and the students need to mark up its essays and point out where ChatGPT is wrong and correct it.

My friend is in university and taking a history class. The professor is using ChatGPT to write essays on the history topics and as the...

reddit
Feb 21, 2023 at 3:29pmWeb
Show threadDavid Fi&erFeb 21, 2023
@cstross The 21st C version of "1066 And All That."
Show threadJosh Ourisman Feb 21, 2023
@cstross this seems like a fantastic teaching tool in several dimensions.
Show threadEllie 🐍 🎄​Feb 21, 2023
@cstross One of my professor friends, back when Wikipedia was new, assigned her Humanities 101 class the task of creating a plausible but fake Wikipedia page and keeping track of how long it lasted, as a similar lesson in that site's trustworthiness as a source.
Show threadErgative AbsolutiveFeb 21, 2023

@noelle @cstross When I was a grad student, one of my professors got fed up with students giving the slightly wrong Wikipedia answer rather than the correct textbook answer, and so he projected Wikipedia up on the screen during lecture and changed it in real time so that it matched the textbook.

He still marked the Wikipedia answers on the homework as incorrect.

Show threadKibrikaFeb 21, 2023
@ergative @noelle @cstross After she changed it? XD
Show threadJoost van der BorgFeb 21, 2023
@noelle @cstross That sucks. Creates extra work for Wikipedia editors, lowers its usefulness for others. Our professor tasked us with creating / updating a good Wikipedia page for a subject we were learning about. Most of the same learning experience without any adverse effects.
Show threadEllie 🐍 🎄​Feb 21, 2023
@joostvanderborg Are you this charming all the time? Go shit in someone else's mentions.
Show threadLisPiFeb 21, 2023
@noelle @joostvanderborg That's a nice way to (dis)miss the point.
Show threadBenAvelingFeb 21, 2023
@noelle @joostvanderborg he may or may not be charming, but he’s right. Vandalism for educational purposes is still vandalism.
Show threaderisianriteFeb 23, 2023
@noelle @joostvanderborg maybe don’t hold up vandalism of a public resource as something to be proud of and you won’t have people being extremely mild in reply to you.
Show threadPeteFeb 27, 2023
@noelle @joostvanderborg
Have you just got off the plane?
Show threadLaura LangdonFeb 25, 2023

@joostvanderborg @noelle @cstross Yep, this. There’s even an official project for this, complete with scaffolding for both instructors and their students! When I taught a course about mathematical contributions to modern society, my students contributed to relevant articles. One group created the page for Data for Black Lives. 🥳

https://wikiedu.org/teach-with-wikipedia/

Show threadgarpuFeb 21, 2023
@cstross That would be a fun assignment! You'd need to know the material and be able to communicate it.
Show threadWeekend EditorFeb 21, 2023
@cstross -- Perfect. Sooner or later, we'll all need to learn how to play "Spot the Bot".
Show threadAngela Korra'tiFeb 21, 2023
@cstross Wow, that's really kind of brilliant. Way to go to that professor!
Show threadPhiala Sarah GosleeFeb 21, 2023
@cstross ChatGPT will write plausible computer code that often runs but may not give the correct answer. I have colleagues making students generate, debug, and explain that code.
Show threadLanda Feb 21, 2023

@phiala @cstross a colleague and I played with it in that regard. Sometimes when it’s told it’s wrong, it will try to fix the problem, generating even worse bugs.

Once it insisted on being right, confabulating a quote that looked like it came from the actual language documentation, used all the right words and phrases but was completely wrong in the same way the program it wrote was.

I wonder if it invented that or if chatgpt itself fell victim to a form of this: https://git-man-page-generator.lokaltog.net/ 😁

Git man page generator

Create an infinite amount of straightforward and readable git manual pages.

Show threadPhiala Sarah GosleeFeb 21, 2023
@cstross @Landa I’m fascinated by the fake citations. Another colleague actually got a request for a reprint, for a plausible paper that chatGPT completely fabricated.
Show threadGeoJoeKFeb 26, 2023
@cstross @phiala Several of my colleagues and I use it to learn libraries that are poorly documented / help troubleshoot issues with our own code. While it may occasionally be inaccurate and not write the best code, it’s a whole lot less toxic than searching and asking stack exchange.
Show threadKatzedecimalFeb 21, 2023
@cstross Chaotic Good
Show threadvarx/socialFeb 21, 2023

@cstross On the one hand, that's awesome. Great way to teach the unreliability.

On the other hand, I can't stop thinking about how making wrong essays was possible before ChatGPT, and the necessity of this exercise in particular is only due to the existence of ChatGPT. It's partly solving a problem that didn't exist before the "solution" was presented to the world. :-)

But it's very clever for sure, and I still like it.

Show threadM.S. Bellows, Jr.Feb 21, 2023
@cstross This is brilliant. Brilliant.
Show threadfelix (grayscale) 🐺Feb 21, 2023
@cstross Yeah I had this idea too last year, and tried to nudge some teachers toward it. I expect many people have thought of it, and I'm glad to see that it seems to work ok
Show threadAntonio YonFeb 21, 2023

@cstross It would be interesting if they could use that kind of feedback to train the next iteration of the ChatGPT LLM.

The fact that they’re *not* using current generation content to train their next model tells you all you need to know about their confidence in the model.

Show threadGerhard SchröderFeb 21, 2023

@pallenberg @Heddergott @Ruhrnalist @breitenbach

:D

Show threadHeddergottFeb 21, 2023
@Gerhard_Schroeder @pallenberg @Ruhrnalist @breitenbach Sollte ja eigentlich ein No-Brainer sein, so mit den neueren digitalen Herausforderungen umzugehen, um neben der Fach- zugleich die zeitgemäße Digital- und Anwendungskompetenz zu vermitteln. Das kann man von Lehrenden erwarten - ist aber leider kein Standard im Jahre 2023… Umso begrüßenswerter ist das Beispiel!
Show threadKai Rüsberg Feb 21, 2023
@Heddergott @Gerhard_Schroeder @pallenberg @breitenbach ich glaube viele Lehrende haben da erst mal Ängste und wissen nicht, wie sie das in ihre jahrealten Abläufe einbauen können.
Ich sage das ohne jede Häme.
Show threadHeddergottFeb 21, 2023
@Ruhrnalist @Gerhard_Schroeder @pallenberg @breitenbach Ich meine das auch ohne Häme, ist eher eine realistische Betrachtung - es is‘ ja, wie’s is. Dennoch könnte man das anno 2023 erwarten, weil das m.E. das generelle Mindset tangiert. Zur Wahrheit gehört aber eben auch, dass die Rahmenbedingungen oft individuelles Engagement einbremsen - ob‘s die eigene (Schul-)Leitung ist, Personalschlüssel und Budgets oder bei uns die jeweilige länderspezifische Bildungspolitik.
Show threadKai Rüsberg Feb 21, 2023
@Heddergott @Gerhard_Schroeder @pallenberg @breitenbach Die Realität ist aber auch, dass in der Lehre oft prekäre Beschäftigungen üblich sind (WisZeitVertrG + Lehrauftrag), die das nicht als Hauptbeschäftigung machen oder Profs, die oft eher widerwillig ihr Deputat erledigen.
Show threadHeddergottFeb 21, 2023
@Ruhrnalist @Gerhard_Schroeder @pallenberg @breitenbach Auch das - das ist die soziale Wirklichkeit (hinter Begriffen wie „Personalschlüssel“), die massiv Motivation nimmt. Es ist einfach versäumt worden, digitale Kompetenzen in die Lehrer:innen-Ausbildung beizeiten zu integrieren. Auf das Engagement einzelner darf man da nicht setzen müssen, wir wissen ja, dass die sich Engagierenden am Ende eben oft nicht belohnt werden.
Show threadPatrick BreitenbachFeb 23, 2023

@Heddergott @Ruhrnalist @Gerhard_Schroeder @pallenberg

Es zeigt doch interessanterweise auf, was wir bei Schule eher in den Vordergrund stellen. Weniger das Lernen und Wissen an und für sich sondern eher die Kontrolle darüber ob jemand nach einer bestimmten Art & Weise, in einer erwünschten Menge, Geschwindigkeit und Machart (Auswendiglernen) "richtig" gelernt hat. Für mich zeigt es eher die Absurdität der Institution Schule und ihr Selbstverständnis auf.

Show threadJenniferFeb 21, 2023
@jbigham related to our discussion yesterday, that could be an alternative way of doing it.
Show threadChina Law TranslateFeb 21, 2023
@cstross A guest on NPR today was essentialy suggesting this tactic, with the added element of asking students to generate the questions for Chat GPT in attempting to get accurate information from it, and explaining what parts the AI seized on in creating responses etc.
Show threadKevin Granade has movedFeb 21, 2023
@cstross This reminds me of my first experience with a real history class. I was assigned an article to review and I was incredibly nervous to turn it in because my conclusion was that the author was incredibly biased and projecting his own views onto the subject matter. I got full marks because that was the point of the exercise.
A very eye opening experience about the nature of history research.
Show threadQuinn NortonFeb 21, 2023
@cstross fuckin cool man
Show threadeswillwalkerFeb 21, 2023
@cstross I hope they are not letting chatGPT know about their corrections!
Show threadDr Micha CampbellFeb 21, 2023
@cstross @Andbaker 👀
Show threadSeaFury ALTFeb 21, 2023
@cstross yup. This is what we are going to do to own students this semester 😬
Show threadJohn LuskFeb 21, 2023
@cstross
Yup.
@delong
Show threadTonya Marie 🏳️‍⚧️Feb 22, 2023
@cstross That is brilliant!
Show threadnadin brzezinskiFeb 22, 2023
@cstross @jackyan jaysus
Show threadGlenn Rempe ✅Feb 22, 2023
@cstross Promote that professor.
Show threadDimonBFeb 22, 2023

@cstross @isagalaev FYI https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb

May be essay will be better if ask to check facts chatgpt )

openai-cookbook/Question_answering_using_embeddings.ipynb at main · openai/openai-cookbook

Examples and guides for using the OpenAI API. Contribute to openai/openai-cookbook development by creating an account on GitHub.

GitHub
Show threadKLBFeb 22, 2023
@cstross genius way to turn the tables.
Show threadRon MillerFeb 22, 2023
@cstross Agreed. This is a great use of the tech to embrace it, while using it as a teaching tool.
Show threadRev. GothAliceFeb 22, 2023

@cstross 🤔 And is this training overwatch being fed back into ChatGPT? Hard to get better training data.

Are these students being used?

Show threadPam KrengelFeb 22, 2023
@cstross This is spectacular. There are quite a few creative educators who are embracing it. Teacher and student learn together, experimenting to understand its current capabilities and limitations. That's a beautiful thing. As they learn, hopefully, continually iterate, and evolve their approach, to know how best to leverage it in the future.
Show threadmvc1095 (they)Feb 22, 2023
@cstross this is brilliant!
Show threadAccount: ComputersFeb 22, 2023
@cstross In before the professor feeds the corrections back to ChatGPT for gamification points.
Show threadAardvarkSagusFeb 22, 2023
@cstross Nice! I am taking a class where I use ChatGPT to generate a basic framework, but then I go through and correct the errors and make the language mine before submitting it. It gets me from 0-1, and I take it from 1-10
Show threadBryan AlexanderFeb 22, 2023
@cstross This is one pedagogy I'm seeing in writing classes as well.
Show threadKevin RussellFeb 22, 2023

@cstross

Now do it with Republicans

Show threadFlexi Bell Feb 22, 2023
@cstross
And about zero work for the professor.
That's what I call brilliant.
Show threadChurch of JeffFeb 22, 2023
@cstross
Show threadJoanne JacobsFeb 22, 2023
@cstross oh I love that :)
Show threadTheo FrancisFeb 23, 2023
@cstross The nice thing about that approach: If you have 20 students, ask ChatGPT to generate 20 different essays of the requisite length on the topic. Then each student gets his or her own. If you know the topic well enough, you can probably still grade them about equally fast (and it's maybe less dull). Only risk is that some essays are wronger than others.
Show threadJamesLaPlaineFeb 23, 2023

@cstross @anildash Ethan Mollick just wrote a great article about this. I also loved his analysis of the type of engagement with ChatGPT that yielded the best results.

https://open.substack.com/pub/oneusefulthing/p/my-class-required-ai-heres-what-ive

My class required AI. Here's what I've learned so far.

(Spoiler alert: it has been very successful, but there are some lessons to be learned)

One Useful Thing
Show threadMike DimmickFeb 23, 2023

@cstross I could swear that I read this idea in @bretdevereaux 's recent blog about ChatGPT. (It's a long article and I can't see the suggestion in a quick skim.)

It's possible that he's the professor in question - or just that whoever it is has read that!