Oto KalábFeb 16
Martin Fleischmann
I am teaching data science, with exam composed of a Jupyter Notebook containing a complete analysis (see https://martinfleischmann.net/sds/course_information/assignment.html). You can guess how the submissions look these days... So, how are you doing courses like this? I am not willing to mark LLMs.
Assignment – Spatial Data Science for Social Geography

Feb 15 at 8:18pmWeb
Show threadFrancisco Rodriguez-SanchezFeb 15

@martinfleis

We have a similar written assignment but a week later they have to present their work: 5 minutes presentation followed by questions, some like 'what's the code doing here?' Or Why did you choose that implementation?

Most students are using AI but most also make the effort to understand their code, as they will have to explain/defend it later.

I think having them present their work is helpful to promote learning. But of course hard to do with large courses

Show threadMartin FleischmannFeb 16
@frod_san That is what I was thinking about. I have 15 students max each year, so it is manageable. Does it work well? Any feedback from students on it?
Show threadHugo LedouxFeb 16

@martinfleis @frod_san I have 35+ students so doing individual interviews where they explain me their code is not really doable.

So this year I tried this: I randomly select 4-5 submissions, and those have to come to my office for a 10min discussion about their code. They can use a LLM but need to be able to explain what a specific line of code does.

It was a test but it went super: most code looked messy (like pre-LLM era) + the randomness was a funny moment in the class.

Show threadMartin FleischmannFeb 16
@hugoledoux @frod_san That sounds fun. With our numbers (~15)I think I can do that with all.
Show threadFrancisco Rodriguez-SanchezFeb 17

@martinfleis @hugoledoux

Yes if there are many students they can work in pairs, and present together

They agree they learn A LOT doing their project. No pushback on presenting so far. I'd just leave it clear beforehand that they should be able to explain their code (ie. It's not simply getting things done), to avoid misunderstandings

Show threadLeonardo UiedaFeb 16
@martinfleis honestly I've given up on assignments like this unless it's a small class and I can get to know the students individually. For large classes, I've gone back to in class handwritten assignments and exams. I've also had to have all students deposit their phones in a basket during exams. It sucks and I tell them so every opportunity I get. I really hate having to play police when I just wanted to be a teacher.
Show threadKen ButlerFeb 16
@leouieda @martinfleis yes, most of the grade has to come from handwritten exams these days. (I think students should sit in front of a computer and use it and their brain to do their assignments, but to judge by the contrast between my assignment scores and exam scores I am no longer confident they are doing that.)
Show threadLeonardo UiedaFeb 16
@nxskok @martinfleis I have one class where grades still come from code-based assignments but they do them in class and me and the TA walk around all the time so I can see what each one is doing. I've only had one incident of LLM code so far, where the code handed in didn't match what was being done in class. Talking to the student resolved this. But this is a relatively small class with at most 30 students.
Show threadMartin FleischmannFeb 16
@nxskok @leouieda I really wanted to have the class focused on practice over theory. They have other classes covering parts of the theory so switching from practice-oriented exam theoretical and hand-written one is the last option…
Show threadPauloFeb 16
@martinfleis @nxskok @leouieda We use a mix of in-class assignments for essays and the like, and presentations followed by questions for project work. The latter is effective but it takes more time and sometimes difficult to organize/schedule in the week we normally have exams at the end of the period.
Show threadLeonardo UiedaFeb 16
@ecodiv @martinfleis @nxskok one way to do it is to allow LLM use but set different criteria if thats the case. I read somewhere (can't find link now) to require disclosure, prompts, and a discussion of biases and caveats in case an LLM was used and then having different rubrics for LLM answers. I've been thinking of trying that next semester.
Show threadMartin FleischmannFeb 16
@leouieda @ecodiv @nxskok I think I've read that as well. I have until September to figure it out. Will be collecting ideas.
Show threadMarekFeb 24

@martinfleis @leouieda @ecodiv @nxskok
Might be this blog post?

https://ploum.net/2026-01-19-exam-with-chatbots.html

Giving University Exams in the Age of Chatbots

Giving University Exams in the Age of Chatbots par Ploum - Lionel Dricot.

Show threadPauloFeb 16
@leouieda @martinfleis @nxskok We're experimenting with that as well, in a series of classes dedicated to date science and, since this year, AI. We'll yet have to see how that is going to work out in the longer run.
Show threadKen ButlerFeb 16
@leouieda @ecodiv @martinfleis I think this could be a recipe for disaster, because you will get students claiming they did it out of their own brains (to get the easier rubric) and you won't be able to prove them wrong.
Show threadPauloFeb 16
@nxskok @leouieda @martinfleis Do you mean giving them the choice? You might be right. In our case, we use only one rubric. There are also courses where students are required to develop or choose their own criteria, but we introduced that before the whole AI thing and for other reasons (and I’m still a bit on the fence about this approach, again, for reasons unrelated to AI).
Show threadLeonardo UiedaFeb 16

@ecodiv @nxskok @martinfleis I found the original post on this by @ploum https://ploum.net/2026-01-19-exam-with-chatbots.html

In that case, it was an in class exam so it works better to let them have a choice without the possibility of cheating like that.

Honestly, I'm inclined to trust students after I explain to them why things are this way and the consequences their choices will have on their future skills. So far, it's mostly worked out for me at least.

Giving University Exams in the Age of Chatbots

Giving University Exams in the Age of Chatbots par Ploum - Lionel Dricot.

Show threadLeonardo UiedaFeb 16

@ecodiv @nxskok @martinfleis @ploum

If the idea is that when using LLMs we'll consider different things when marking, then I don't think many students would have reason to think the non-LLM option will get an easier rubric. I can imagine them thinking that explaining when something is wrong is much easier than getting the LLM to do everything to spec and then getting marked on the technical aspects.