RE: https://mastodon.scot/@simon_brooke/116232834837623434

“The researchers suspect that Glassworm—the name they assigned to the attack group—is using LLMs to generate these convincingly legitimate-appearing packages. “At the scale we’re now seeing, manual crafting of 151+ bespoke code changes across different codebases simply isn’t feasible,” they explained. Fellow security firm Koi, which has also been tracking the same group, said it, too, suspects the group is using AI.”

What I don’t get is how this snippet passed code review regardless.

I mean, it’s clearly dodgy and the last line basically meaningless without the code being evaluated.

The real story here isn’t the invisible Unicode characters, it’s the lack of proper code review on code submissions.

@aral I agree that code reviews should generally be "wtf?! eval?!", but it is always good to remind people that an empty looking string is not necessarily really empty.
@aral lol, the eval call not even obfuscated
@aral

I feel like I need to restate this:

An "empty" string in back ticks not ringing any alarm bells.
@aral I agree, I had the same thought, first of all an eval function is super suspicious; "eval is evil", on the other hand what the earth is the const s doing there. And in the name of love what is the reviewer thinking about when someone does a PR with this kind of "functionality". I will check but I think that a simple SAST is going to complain about this PR.
On a second thought maybe the snippet provided by the security researchers is just a non realistic example to illustrate the concept.

@aral adding arbitrary Unicode-high constants (like 0xFE00 and 0xE0100) to text buffers to obfuscate payloads is something that malicious actors have been doing for a long time (on top of tricks like hiding code 10k spaces after the end of a line).

The thing is this is exactly where automated reviews could be useful.

Even a simple grep test to check if your code runs eval() should at the very least trigger a warning in npm.

The presence of non-readable Unicode characters in text files (again, something that grep+sed can easily spot) should also be a big red flag.

And they could definitely train a small AI reviewer to identify common patterns (lines with too many spaces, cheap source code obfuscation techniques or execution via eval() of stuff that comes from arbitrary externally-controlled buffers). It’s not like we’ve never seen these things in the past 3 decades or so, we’d have plenty of material to train a good model.

Extensions uploaded to Mozilla for example go through a similar pipeline of checks before being approved.

The fact that the npm (and even pip) registries don’t run these basic automated checks, and packages are taken down only when shit has already hit the fan, is really disturbing.

@Aral Balkan I thought the same thing when reading this piece. When doing security code audits these kind of decoder functions is pretty much the first thing we look for. And they're pretty easy to spot using standard tools (no, not "AI") as well.
Aral Balkan (@[email protected])

59.3K Posts, 11.3K Following, 46.8K Followers · Social oncologist. I make small things. Unapologetically anti-genocide. Warning: May not fit your taxonomy. From Gaza? If you need to get verified, please go here: https://gaza-verified.org/join/ Want to donate to people in Gaza? Please see https://gaza-verified.org/donate/ My posts are licensed under Creative Commons Attribution-NonCommercial-ShareAlike (https://creativecommons.org/licenses/by-nc-sa/4.0/) 🍉🌻 🏳️‍🌈 🏳️‍⚧️ #SmallWeb #SmallTech #web #tech #privacy #humanRights #personhood #democracy #aral #fedi22 #searchable

Aral’s fediverse server