Christine Lemmer-WebberFeb 23

It's demotivating to think that:

- LLMs aren't good at producing original / novel work
- You still need experts to advance that stuff
- It will always be slower to move without using LLMs
- Once an innovation is done though, an innovation can always be scooped up by the LLM users
- "Bro why are you doing all this manually, I just vibe coded that in a weekend"

Will it always be this way? It's depressing in the meanwhile, at least.

Show threadChristine Lemmer-WebberFeb 23

In a sense, the decision is somewhat made for us in that we're developing next-generation stuff that LLMs don't know how to auto-code at @spritely. We are working on core infrastructure that needs to be carefully thought about and written. LLMs introduce a lot of errors and aren't good at doing this kind of work on their own.

And the goal was always that our work is there to be lifted from, to spread outward, the way people have long drawn from the well of the MIT / Stanford research labs in CS for decades, but for decentralized networking today

But doing it now, in this way, in this environment, it's just really depressing and demotivating.

Show threadmccFeb 23
@cwebber @spritely I mean the problem as I see it is: The people who primarily benefit from the work aren't paying for it, and there's no way to get them to contribute back ("licenses" no longer exist). So the art can only be extended by individual humans expending their savings or going into personal debt. (In theory basic research could additionally be funded by corporations, but since people who care about the art exist as a resource to be exploited, there is no reason for them to do so.)
Show threadmccFeb 23
@cwebber @spritely This is similar to the problem I have making video games: Some portion of my audience will pirate my work. Technically that doesn't harm me, *but* if *everyone* pirates the game then I don't get any money and I don't get to keep making games. I decide I don't care because not everyone pirates games and *some* of the people playing the game will pay for it. LLMs, for code, sets up the possibility the entire audience will be pirating the work. Which is wild since my code is MIT
Show threadmccFeb 23

@cwebber @spritely This said, I want to give you the flipside to the process you're describing: I am currently creating a small programming language which exists for no purpose except for me to make games for the Game Boy and NES. When I look at my language, I think: *An LLM user could not use this language, because there is not a sufficient corpus to generate code from¹*. And this sparks joy in me

¹ And a significant portion of the corpus is testcases designed to fail

Show threadDaniel V.Feb 23
@mcc @cwebber @spritely a painstakingly pre-poisoned dataset 🥰
Show threadmccFeb 23

@dvandal @cwebber @spritely I think it is important to write test cases and I think it is important your test cases test your failure modes!

:3

Show threadDaniel V.

@mcc @cwebber @spritely I work in QA, so my job is to test those failure modes. (Automatically and at scale to boot!)

And you are right! It is important to test those cases

Feb 23 at 4:42pmWeb