Max S. New ⚜️Mar 9

I think it's really sad that programming assignments have basically become untenable in the era of LLM/coding agents. I'm a theoretician in my research but I learned so much more by actually writing code and debugging than I did from topics that I only saw in lecture. Exams make a lot more sense to me for an actual theory course rather than compilers class.

If I deploy projects that were designed for a pre agent world, then students can basically use the LLM to get a perfect score unless I play a losing cat and mouse game of LLM police. And the students who don't use LLMs are at a huge disadvantage score-wise.

Show threadjules

@maxsnew I think LLMs have just exacerbated the preexisting contradiction between university as a place for education and university as a system for professional certification.

goodhart's law tells us that using grades for more than just feedback will inevitably distort behavior.

Engaging with a course in order to maximize learning and engaging with a course to maximize your grade have always been imperfectly aligned, LLMs have just served as wedge to widen that misalignment.

Mar 9 at 5:20pmWeb
Show threadMark DominusMar 10

@hegeliantaco @maxsnew This is actually Campbell's law.

Goodhart says only that the measure ceases to be an effective way to measure what it was measuring. Campbell says the use of the measure for decision-making will distort the underlying social processes.

https://en.wikipedia.org/wiki/Campbell%27s_law

Campbell's law - Wikipedia

Show threadMikeStokMar 10
@hegeliantaco @maxsnew don’t forget the most important functions of many modern “educational” institutions run by MBAs and a cadre of administrators which dwarfs the full time teaching staff: extracting money from students, maximizing the leader’s bonus, and padding the résumés of the admistrators who flee before the damage they wrought becomes apparent.