tired_n_boredHey we solved software development, no need to learn programming anymore (Claude's source code leak)
puchaczykSo the innovation in Claude was to write 95% of the prompt for the user and make you use like 10k tokens
The problem is that words don’t have meaning in the genAI field. Everything is an agent now. So it’s difficult and confusing to compare strategies and performance.
Claude Code is a pretty solid harness. And a harness is indeed just prompts and tools.
JackbyDev
HamartiogonicJust write good code. It’s as simple as that, right?
renzhexiangjiao“make no mistakes”
rozodruAsking Claude to not hallucinate is like telling a person to not breathe. it’s gonna happen, and happen conistently.
I think the important bit to understand here is that LLMs are never not hallucinating. But they sometimes happens to hallucinate something correct.
James R KirkThis fact of how LLMs work is not at all widespread enough IMO.
driving_crooner“Include no bugs”
They are spending thousands of dollars in tokens and write the most complicated prompts in order to avoid writing good specifications.
That may actually work a little?
I mean, it scraped the entirety of StackOverflow. If someone answered with insecure code, it’s statistically likely people mentioned it in the replies meaning the token “This is insecure” (or similar) should be close to (known!!) insecure code.
addieI was part of that OWASP Application Security Verification Standards compliance at my work. At a high level, you choose a compliance level that suitable for the environment you expect your app to be deployed in, and then there’s a hundred pages of ‘boxes to tick’. (Download here.)
Some of them are literal ‘boxes to tick’ - do you do logging in the proscribed way? - but a lot of it is:
- do you follow the standard industry protocols for doing this thing?
- can you prove that you do so, and have protocols in place to keep it that way?
Not many of them are difficult, but there’s a lot of them. I’d say that’s typical of security hardening; the difficulty is in the number of things to keep track of, not really any individual thing.
As regards the ‘have you used this thing in the correct, secure way?’, I’d point my finger at something like Bouncy Castle as a troublemaker, although it’s far from alone. It’s the Java standard crypto library, so you think there would be a lot of examples showing the correct way to use it, and make sure that you’re aware of any gotchas? Hah hah fat chance. Stack Overflow has a lot of examples, a lot of them are bad, and a lot of them might have been okay once but are very outdated. I would prefer one absolutely correct example than a hundred examples have argued over, especially people that don’t necessarily know any better. And it’s easy to be ‘convincing but wrong’, and LLMs are really bad in that case. So ‘ticking the box’ to say that you’re using it correctly is extremely difficult.
I see the Claude prompt is ‘OWASP top 10’, not ‘the full OWASP compliance doc’, which would probably set all your tokens on fire. But it’s what’s needed - the most slender crack in security can be enough to render everything useless.
Writing all these prompts almost seems like a more time-consuming thing than actually programming the software.
Sundray
underisk
Fargeol
So I don’t know if all the other replies are pretending to be stupid, but the shown prompt is not stupid.
If you include stuff like that section in your prompt, then it has been shown that the AI will be more likely to output secure code. Hence of course the section should be included in the prompt.
If it looks stupid but it works, then it is not stupid.
ChaisFirstly, it can work and still be stupid.
Secondly, since the chat bot is more likely but not certain to write secure, bug-free code, it does not in fact work and is therefore, by your own reasoning, stupid.
But so is asking a chat bot for code to begin with, so there wasn’t ever really a way around that.
Secondly, since the chat bot is more likely but not certain to write secure, bug-free code, it does not in fact work and is therefore, by your own reasoning, stupid.
But so is asking a chat bot for code to begin with, so there wasn’t ever really a way around that.
since the chat bot is more likely but not certain to write secure, bug-free code, it does not in fact work
Humans are not certain to write secure, bug-free code. So human code is useless, by the very same metric?
What kind of “logic” is that?
Rose“Don’t put in any of the Top 10 vulnerabilities. But if you put any from the 11th place and down, that’s okay, I don’t even know what those are.”
(Also, getting flashbacks from Shadiversity plugging “ugly art” and “bad anatomy” in the negative prompt as he was no doubt silently wondering why it didn’t work)
“In other news, popularity of attacks against OWASP vulnerabilities #11-20 rose sharply.”