Mastodawn

Georg Weissenbacher Mar 5

@krismicinski @shriramk @jfdm @csgordon @lindsey @jeremysiek we have been claiming for decades that we are not just educating coding monkeys, so it shouldn't really matter that LLMs can now do all the coding. As far as I see it, it's still necessary to identify and clearly formulate verifiable requirements and specifications, come up with a modular design, and verify the whole thing, because I still believe the ultimate responsibilty lies with the developer. So students still need to understand the fundamentals. But yes, it has become much harder to check *at scale* whether they actually grasped them.

Shriram Krishnamurthi Mar 6

@GeorgWeissenbacher @[email protected] @jfdm @csgordon @lindsey @jeremysiek
Yes to most of that. I think it's not that hard to assess if that is what people were always assessing that.

I actually disagree w/ your opening comment. Most intro CS educators will say (and have said), "I don't teach programming, I teach *problem solving*" (whatever the fuck that is). My response is, "great, this should be your liberation! Programming got easy, what are your «problem solving» ideas?"

Tony Garnock-Jones Mar 6

@shriramk @GeorgWeissenbacher @krismicinski @jfdm @csgordon @lindsey @jeremysiek ... did programming get easy? Can one be said to be programming if one asks someone else (or an LLM) to write a program for you? Or is some other kind of (not- or not-quite-programming) interaction going on?

Shriram Krishnamurthi Mar 6

@tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @lindsey @jeremysiek
I very much think of what I'm doing with Claude Code as a kind of programming — indeed, the kind of programming I always wished I could do! But if it makes you happier to use a different term for it (not "vibecoding", that has too many specific connotations and is definitely not how *I'm* doing things), and it's *useful* to have that other term…that's fine by me. I guess my slogan is: "Philosophy…but not too much".

Lindsey Kuper Mar 6

@shriramk @tonyg @GeorgWeissenbacher @krismicinski @jfdm @csgordon @jeremysiek This comment made me realize something about myself: this is *not* a kind of programming I always wished I could do. I really only like programming because I like manipulating formal systems. That might explain a lot about why this kind of programming doesn't appeal to me, aside from all the bad externalities.

Shriram Krishnamurthi Mar 6

@lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek
One thing I've learned over the past few months is that there seem to be two kinds of "programmers" even in the very rarified space of "profs with a PhD in PL/FM/…": those who really like code and those are indifferent to it in light of other aspirations. Until now we had no way of telling ourselves apart. Now we do. It's been really interesting, revealing, and fun to see this split.

John Regehr Mar 6

@shriramk @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek based on available evidence so far (I'm enjoying the hell out of LLM-assisted programming) I'm not as much of the first kind as I had previously supposed.

John Regehr Mar 6

@shriramk @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek like Csmith had sort of slowly decayed over the last 12 years and last weekend I went and did a bunch of much-needed cleanup and modernization. would I have done that w/o LLM support? absolutely not

Shriram Krishnamurthi Mar 6

@regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek
This is the key. The things that were never being done. Maybe it's a prof thing? I remember having all the time in the world to code, as a grad student, and now I have virtually none, and building the things I want involves not just thinking but reading 50 APIs and using 5 languages with 3 syntaxes and 2 partridges in a pear-tree.

John Regehr Mar 6

@shriramk @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek right, I hadn't realized how bottlenecked I was by lack of student + my own time. like I have one million ideas and now I can pursue 0.0007% of them instead of 0.0003%. or whatever.

LLM coding is perfect for profs-- we're time-limited experts and we're mostly not expected to produce really awesome code anyhow.

Shriram Krishnamurthi Mar 7

@regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek
In particular, it's letting me make a lot more progress on my 1/10th-baked ideas, helping me take them to at least quarter-baked ideas, before I ask a student to think about them. I've definitely had situations before where the lack of that prototyping meant I would give a poor, unsuspecting student some "brilliant" idea and a semester later find out it didn't go anywhere. ↵

Shriram Krishnamurthi

@regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek
I feel like I can be a more responsible advisor now — at least giving it a whirl on my own time and seeing how it pans out. If I'm not willing to do even that, then I certainly shouldn't be foisting it on a student, for whom failure is infinitely more problematic. ↵

Shriram Krishnamurthi Mar 7

@regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek
One "problem" I've had is that the prototypes emerge so quickly that the blocking function is my thinking. I'm working on a new PL/medium(!) that I've never been able to interest a student on; and I've quickly hit the limits of how far I'd thought things through. Now the repo tab just sits there in my shell staring at me accusatorially, whispering, "Okay, what next, smart guy?" Gulp. ↵

Shriram Krishnamurthi Mar 7

@regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek
Papert, Kay, etc. used the phrase "tools for thought". (@tonofcrates is teaching a course by that name!)

This feels like a new tool-for-thought. Including exposing both the weakness and incompleteness of my thought, which is what a good tool ought to do.

As a PL person, I'm excited to be able to rapidly prototype a PL and actually *use* it, not just reason through calculi (a different tool for thought).

Jeffrey P. Bigham 🔥🔥Mar 7

@shriramk @regehr @lindsey @tonyg @GeorgWeissenbacher @krismicinski @jfdm @csgordon @jeremysiek @tonofcrates i've had a similar experience, but have found it really awkward to use these tools -- what am i supposed to do in the ~(small. number of minutes) during which claude is doing stuff. think? follow along? mostly i go and check mastadon lol

Shriram Krishnamurthi Mar 7

@jbigham @regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek @tonofcrates
I really like Jakob Nielsen's summary of 30 years of UX research on 3 levels of interactivity:
0.1 second = instantaneous
1.0 second = uninterrupted
10 seconds = keeps attentionª
ª still true?

Anything beyond that, and your brain wanders. These things have the *worst* response time, returning in minutes rather than a day.

That's why we have *three* social media.
https://www.nngroup.com/articles/response-times-3-important-limits/

Response Time Limits: Article by Jakob Nielsen

How users react to delays in a user interface, whether website or application. The 3 main response time limits are determined by human perceptual abilities.

Nielsen Norman Group

Jeffrey P. Bigham 🔥🔥Mar 7

@shriramk @regehr @lindsey @tonyg @GeorgWeissenbacher @krismicinski @jfdm @csgordon @jeremysiek @tonofcrates

i think the big opportunity, which is hard, is how to usefully keep people attending with these things. i'm starting to see work where humans are still in the loop. but, that requires a lot more focus on updating the human's mental model (need way more than just following along the chat), and then how would you usefully intervene?

Jeffrey P. Bigham 🔥🔥Mar 7

@shriramk @regehr @lindsey @tonyg @GeorgWeissenbacher @krismicinski @jfdm @csgordon @jeremysiek @tonofcrates regardless, most of the steps are pushing on the 10 second mark, and when they are just coming in sequence every 5-10 seconds, it's super boring. so many of hte updates are meaningless anyway, how many synonyms for "thinking" do i really want to see flash on the screen lol

Shriram Krishnamurthi Mar 7

@jbigham @regehr @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek @tonofcrates
To me a really exciting open research question is how can we keep humans in the loop in a way that their work is
- minimal/moderate
- meaningful

Either one is easy. Security alerts were minimal, but useless because not meaningful. "Review all the generated code" is (maybe) meaningful, but not moderate.

In specialized sub-domains we have some answers. Don't yet know how to generalize.

John Regehr Mar 7

@jbigham @shriramk @lindsey @tonyg @GeorgWeissenbacher @[email protected] @jfdm @csgordon @jeremysiek @tonofcrates more AI sessions

Tony Garnock-Jones Mar 9

@shriramk @regehr @lindsey @GeorgWeissenbacher @jfdm @csgordon @jeremysiek @tonofcrates That's so interesting that you see it as a tool for thought. To me, it has felt predominantly like a tool for escaping thinking. One delegates the thinking. And then, having not done the thinking, has trouble understanding the result in anything other than a superficial way... But it's definitely a tool for achieving some effect in the world! I just wouldn't characterise that effect as "thought". Perhaps it's a kind of lever/fulcrum: some kind of force or distance advantage, more like a (mechanical) machine than like a mental augmentation??

Will Crichton Mar 9

@tonyg @shriramk @regehr @lindsey @GeorgWeissenbacher @jfdm @csgordon @jeremysiek

A simple, albeit terribly incomplete, analogy is that an LLM is a fancy search engine. Like any search engine, it can hinder thinking (e.g. searching for solutions to a homework problem, plagiarizing sources for an essay) or support thinking (e.g. facilitate debugging, surface unexpected sources).

Shriram Krishnamurthi Mar 9

@tonofcrates @tonyg @regehr @lindsey @GeorgWeissenbacher @jfdm @csgordon @jeremysiek
As someone who thinks about metaphorical and analogical thinking, I'd say this is a terrible analogy in general and a fantastic analogy in this specific case and for this audience. (-:

That is, it fails very badly as an analogy for understanding mechanism, but it works very well as an analogy for understanding use/effect. I hadn't really thought about that distinction before: analogies for purposes.