Mastodawn

Filippo Valsorda

RE: https://wetdry.world/@16af93/115961732893013803

Because not using AI tools for what they excel at will produce less secure code.

For example, they are great at debugging (https://words.filippo.io/claude-debugging/), they can find real issues in code review, they know more math than me or most of my colleagues, and they can write static analyzers I would have never had the time to write myself.

@filippo I think the "they can build [include any quality-increasing measure] I would never have had the time to write myself" is a very underappreciated effect of LLM-assisted development.

@filippo It's sort of the opposite of vibe coding.

Can Acar Jan 26

@filippo I am sorry, but a cryptographer saying something like "they know more math than me" only tells me that the cryptographer in question does not know how those things work. Please do not underestimate yourself or overestimate the capabilities of a text generator that happens to be have ingested tons of stolen human generated mathematical text that it stitches together (or quotes verbatim without attribution) to look like an answer.

Filippo Valsorda Jan 26

@canacar I know my capabilities (and their limits!) thank you very much, and your description suggests you have not seriously tried a state-of-the-art model for more than five minutes.

Load up Claude with Opus 4.5, ask it to reason about stuff you know the right answer for, and get back to me.

I am good at combinatorics/probabilities (IMO Bronze medal), and it still helped me do the analysis for the recent bruteforce of test vectors I did.

Can Acar Jan 26

@filippo the "reasoning" is a series of RAG queries, which in turn are web searches or agent outputs that then get added to the context, with no additional component of "understanding" or "knowing" or "reasoning". Just text generation with more context which may or may not be correct. Yes, they are helpful if you can verify the output and they speed things up if you can easily identify and discard incorrect outputs

I am not a developer. I am on the other side, dealing with summaries devoid of content or originality and and increased workload because people think that these things are like a fellow developer that "knows" or "learned" something just because they did it correctly once.

In that, I support your effort pointing these tools to better patterns, but refuse to anthromorphize it.

Filippo Valsorda Jan 26

@canacar "reasoning" is about using longer outputs to produce better final results, it has nothing to do with RAG and little to do with extra context.

You don't have to anthropomorphize them, but you are doing yourself a disservice by thinking about themselves in excessively simplified terms which seem to describe Markov chains more than LLMs.

The Anthropic blog has a lot of great research if you want a more realistic mental model, or again you can try them.

Daniel Spiewak Jan 26

@filippo @16af93 It’s almost as if they’re a tool like any other: applying that tool correctly creates better outcomes than if you ignore it, just as applying it incorrectly creates worse outcomes.

Lambda Crime [The Möth Society]Jan 26

@djspiewak @filippo a tool of ethically at best dubious nature, but then again the torment nexus is just a tool too

Filippo Valsorda Jan 26

@16af93 @djspiewak ^ a social media post sent over the internet using a device made in China, each a tool of ethically dubious nature

Lambda Crime [The Möth Society]Jan 26

@filippo @djspiewak there is no ethical consumption under capitalism, but at least the phone that sends these posts has been chosen to reduce the harm caused, further a phone is nearly universially mandatory to use to participate in society - so not that much of a gotcha as you think

Daniel Spiewak Jan 26

@16af93 @filippo There's definitely a lot of dubiousness going on, I agree. As someone who has about two and a half decades of OSS corpus which has been stolen and swirled around in the bowels of these things, I'm very uncomfortably familiar with the contradiction here. But my discomfort or lack thereof isn't going to change anything. I can disadvantage myself and those who rely on my production, but I can't really affect the problem at its root.

Lambda Crime [The Möth Society]Jan 26

@djspiewak @filippo usage gives legitimacy to ai, and legitimacy allows further exploitation and laundering of exploitation

personally these tools produce too much harm on every level to be given legitimacy (see the recent trends with ai psychosis, people being diluted by these overly agreeable models to the point of being emotionally disturbed by models changing their behavior (see the ai gf crowd), and curl needing to stop their bug bounty program because of ai swamping them.with bogus reports)

@filippo @16af93 @djspiewak This toot is so disappointing. There are many reasonable things one could say about llm ethics but "your ethics argument is invalid because you are also using unethical things!!!" feels not in good faith.

You're right that there's no ethical consumption under capitalism. That doesn't mean we shouldn't care, or try to do better.

Filippo Valsorda Feb 16

@danvolchek @16af93 @djspiewak in context, my three weeks old post is reinforcing that LLMs are indeed tools like any other (including being subject to valid ethical debates, although that was not the topic of the thread). Why are we playing delayed-action short-form context collapse?

felixlinker Jan 26

@filippo Using AI to generate test cases for my current project, and it's just so valuable. Would have never done that on my own. Especially because the cost of setting up the entire testing pipeline is often quite high for me (remembering or learning a testing framework).

I use QuickCheck for Haskell, i.e., I test invariants on my code with random inputs. It's so nice to describe the expected invariants in natural language and have them almost ready to test.

Ayush Agarwal (आयुष अग्रवाल)Jan 27

@filippo I'll try to keep my post short but I'm curious, do you feel the advantages of using LLMs outweigh the disadvantages I can think of, namely:

becoming potentially dependent on a proprietary tool for finding issues in code review, learning math, and writing static analyzers? These tools are probably subsidized at the moment, would you pay substantially more money for them if they raise their prices?
these tools are trained on code written by other people without their consent and without any sort of attribution and although some people may gloss over that fact because of how they feel about copyright laws, these models also DDoS websites, small independent git forges, and small businesses who do not always have the resources to defend themselves against such abuse. Do you feel comfortable with directly or indirectly contributing to such DDoS attacks by using LLMs?
these tools are also responsible for the deterioration of personal computing by concentrating the availability of personal computing hardware in the hands of a few big tech companies for their data centres, not to mention the concentration of knowledge as a result of the previous point I mentioned because people will either find shelter in big tech platforms which can handle DDoS or quit creating knowledge publicly. Is using LLMs worth losing access to personal computing resources and human knowledge? I was considering assembling a new desktop this year but those plans are now on hold indefinitely. I don't think whataboutism and moral relativism helps here. I don't expect a person involved in the craft of creating useful software to fight the battle of ethical working conditions in China or care about responsible sourcing of silicon and lithium. A person does what they can with the limited resources they have in life. I do, however, feel that someone in the open source ecosystem cares about the health of that ecosystem and that is definitely getting affected negatively because of LLMs. Should everyone wall themselves off behind CloudFlare?
do you feel you can always draw a clear line where you won't use an LLM? Even though some people call LLMs "just tools", they don't behave like a software tool with a specific purpose (a text editor or a compiler, for example). They also can't be relied upon like a trusted human partner and even in such cases, a human is granted trust in specific cases after they earn it in some manner. The public usage of such tools will inevitably bring suspicion about the lack of human review because of how predominant that pattern is.

Filippo Valsorda (@[email protected])

@[email protected] @[email protected] ^ a social media post sent over the internet using a device made in China, each a tool of ethically dubious nature

Mastodon