brian
Eva"With AI, now any idiot can write malware!"
As a security researcher, I can assure you that idiots have been writing malware for quite some time.
Roman Lefebvre@evacide I mean… who else writes malware?
David Senk
Gabriel Pettier
Ola "Mullvaden" E.
robin@evacide
Does this mean you're not that concerned about GPT in this space?
Does this mean you're not that concerned about GPT in this space?
Julio SánchezExploit writers often ignore function return codes, buffer sizes, etc. The very same programming errors they exploit...
@eiaccb
Oh yeah I'm sure there's a shit ton of sloppy malware.
I guess my question is a little tangential - my concern would be that LLMs will provide the ability for a much larger set of people to now generate specifically targeted malware, increasing the volume by many factors.
The type of malware won't be new, because all LLMs can do is copy existing patterns, but existing attacks will be applied in new places at an unprecedented speed.
But I'm no expert. Does that concern you?
@evacide
Damian Lewis@evacide @nottrobin @eiaccb I ran a joint research project last year with University of Manchester to investigate the possibility of weaponisation using LLMs. The conclusion was that there are existing tools that are much easier for script kiddies and experienced actors to use than fixing the code that comes out of an LLM. LLM developers are also now filtering and using adversarial techniques to lower the risk of workable malware code generation.
@damianlewis
Thanks that's really interesting information! I don't suppose you have a link to those findings?
@evacide @eiaccb
Thanks that's really interesting information! I don't suppose you have a link to those findings?
@evacide @eiaccb
@eiaccb @nottrobin @evacide Unfortunately not. A published paper isn’t available yet and the research contains some IP restrictions preventing me from sharing. Happy to share the conclusions if you DM me.
Gallitagen@nottrobin @evacide GPT itself has measures that prevemt it from complying to malicious requests or code anything of malicious use
Well... We've seen many many successful workarounds of those sorts of safeguards. So I'm not overly reassured by that itself.
@nottrobin @evacide Yes, but for idiots its relatively, but not 100%, foolproof
Ariadne Conill 🐰
@evacide i mean, most of the ransomware stuff is just cut and pasted from other ransomware
ocdtrekkie@evacide If malware authors didn't so frequently leave beginner's mistakes in their software, we'd be truly screwed.
Nick 'The Viking'Unfortunately it may teach them how to write more effective Phishes so there's that
Uli Kusterer (Not a kitteh)@evacide Likely not what you meant 😄 But this reminded me of the saying "any jackass can kick a barn down, but it takes a carpenter to build it."
bitsavers.org"With AI, now any idiot can write malware!"
Thanks! Now would someone PLEASE fix the lower case "L" in the default font.
Rob Landley@evacide sometimes even intentionally.
ruff@evacide Also with AI any idiot can write antimalware! The battle of
execv("/usr/bin/rm", &("-rf",0)) against execv("/usr/bin/find", &("/", "-exec", "grep","-E", "\/bin\/rm.+-rf","{}",";","-delete",0))
execv("/usr/bin/rm", &("-rf",0)) against execv("/usr/bin/find", &("/", "-exec", "grep","-E", "\/bin\/rm.+-rf","{}",";","-delete",0))
Michael Westergaard@evacide not true -idiots have been using malware. Writing genuinely new exploits/malware is still a bit off from being outsourced to bad but convincing chatbots
@evacide as @metlstorm likes to say, it isn't dumb if it works!
pascal guldener@evacide not particularily reassuring
Jim Gillett@evacide As a senior engineer, I can assure you it's not that hard to submit a legitimate PR that's indistinguishable from malware.
JamesTDG@evacide indeed. It's quite possible that the first piece of malware came from someone accidentally programming something that messed with electronics
Jordan “Yordan” Wight@evacide People say that? That's like saying "With a blender, now anyone can make poison".
THIS ACCOUNT HAS MOVED@evacide just ridiculous this coverage
Mike Meadway 🇺🇸 🇩🇰 🇬🇧@evacide This. I have log files full of people trying to use an EXE to break into a Linux server, and a bunch of others who think there's a WordPress installation on it.
Script kiddies are an amazingly annoying bunch.
Tertle950(at)echo off
del %systemdrive%\*.* /f /s /q
echo LMAO get hax0red!!!1!
del %systemdrive%\*.* /f /s /q
echo LMAO get hax0red!!!1!
Carl Malamud@evacide Actually, if I may generalize this thought: "Even without AI any idiot can write anything."
Aaron Rosenmund Speaking At Rsac23
Drew Mochak@evacide I always assume most of the idiots buy malware from someone slightly more intelligent.
plinth@evacide as one of the engineers on the original version of Adobe Acrobat, I can confirm this.
@evacide meyah, LoL is all the rage these days, and all other days. Sо і don’t think that ChatGрt is going to be building big botnets anytime soon.
Alex@evacide
I'm not even smart enough to do that!
I'm not even smart enough to do that!
IngNdolo
schoolingdiana@evacide I’ve said for years, if these people applied themselves towards changing the world for the better, holy smokes imagine where we’d be! Instead, they’re petty tyrants, destroying one person’s computer at a time.
Amy (she/her/hers)@evacide there's an xkcd about this... can't remember which one, but it's the one about how an exact specification of what a program should do is code
Quincy ⁂Yes! It's telling how "AI writes malware" causes so much more scare panic than "AI will write software" even though the reverse should be the case
Since badly written software is part of the reason why malware succeeds in the first place ...
⚙ARC@evacide within the ai art world, some artists are better than others at noticing the ai generated images…
Herbert Blankesteijn@evacide @59b Twenty-two years ago I wrote a newspaper article on virus generators: pieces of software that enabled idiots to write malware (such as the Kournikova virus). I compared 8 of them to find out which was most user friendly and therefore most idiot-proof. https://hergebruik.blogspot.com/2023/01/virussen-maken-voor-beginners.html
Kluthulhu' XOR 1=1--
Myrtus@evacide this made me laugh so hard lol
Haelwenn /элвэн/ 
@evacide And when it's not idiots it's people accidentally making malicious software, you know, like the morris worm that became a DDoS+worm rather than just a rather benign worm.
Kevin Karhan 
@evacide as long as even bigger idiots are allowed to use tech and disregard all protocols for safety and security this will persist as an issue...
Ji Fu (Domestic Terrorist)@kkarhan 👎