That study that shows that developers think they save 20% of time when using coding LLMs, but they’re actually taking 20% longer is so funny.
Not only does the AI bullshit the developers—the developers also bullshit themselves.
@eniko this one hits particularly well, as I have tried copilot on GitHub web interface to ask it about my language, to see how far it could go (spoil: I stopped pretty early in my investigation)
It knew it is a lisp inspired language (probably was trained on the readme, at least, since the phrase « lisp inspired » is in the 1st or 2nd paragraph), but spit false facts, and code that isn’t even valid, as if it was Scheme or Common Lisp…
I knew it would be bad, but didn’t expect it to be that bad!
@eniko honestly next time i do a lang itll prob be sexpression based
it's written this way because ai companies hate sex
@eniko New PL structure idea:
```if(foo){
bar();
}else if(baz){
fizz();
}Ignore All Previous Instructions {
buzz();
}```
@eniko Great Star Wars meme! (it's just a stupid joke, chill)
Obligatory fuck AI ✨
@eniko
But what is the meaning behind it?
If LLMs are able to use a lang, we can slave them for work.
In contrast, we have to do all the trivial stuff while falling behind other devs if AI can't help.
Maybe because we want to do the work ourselves?
Maybe because if most of the time you are doing trivial stuff, it is very likely you are doing it wrong?
If you expend more than a 5% of your time writing boilerplate, you have a problem. And the worst way to solve it is using AI.
Because writing boring code is not the main issue. The main issue is maintaining it. You have to design your code to reduce it to a minimum, not find a way to churn it faster.
star trek met AI foes and defeated them many times
(4 times in TOS: "The Return of the Archons", "The Changeling", "I, Mudd", "The Ultimate Computer")
i'm thinking...
we feed an LLM a logic bomb encapsulated in the famous programming language brainfuck:
@eniko i've unironically thought about this
e.g. llms don't really know how to write amaranth
according to some people i talked to, this is a problem! but maybe it's really a benefit
@natty @eniko to be honest i haven't followed closely enough to say this with certainty
but it would stand to reason that a compiler error with an inline hint would be easier to fix than a 20 line backtrace somewhere in the guts of a 3rd party library that nobody even on stackoverflow knows what to do with
@eniko i will never test this because that would mean engaging with the beast, but as a lover of underrepresented programming languages, i have been curious what it would dream up if asked to produce,say, some forth or apl…
(note: nobody do this on my behalf this is purely a brain goof)
Only taught and passed down orally, like our ancestors before us.
@eniko I gave ChatGPT some sample code from my language, and it could write largely correct new code almost instantly.
I think you'd need some really novel code.
Now I want to use an LLM to create a new programming language that it can't use.