BrianKrebs

One of the more interesting graphics I've seen regarding the XZ backdoor is a representation of Jia Tan's commits over time. Notice how the commits in question were done well outside the normal times this user committed code in the past.

Does this lend credence to the notion that somehow the Jia Tan account was hijacked? Maybe. Or maybe it just means the attackers got sloppy at the tail end of a 2 year op for unknown reasons, like they were up against a hard deadline that was tied to something happening IRL.

I'm curious what the prevailing theory is here.

Apr 3, 2024 at 6:24pmWeb
Show threadBrianKrebsApr 3, 2024

I was somewhat able to follow along here, but I got lost a few times. Does this mean we think Libarchive also was also messed w/ by the XZ backdoor bandits?

https://github.com/libarchive/libarchive/pull/1609

Added error text to warning when untaring with bsdtar by JiaT75 · Pull Request #1609 · libarchive/libarchive

Added the error text when printing out warning and errors in bsdtar when untaring. Previously, there were cryptic error messages when, for example in issue #1561, the user tries to untar an archive...

GitHub
Show threadFarce MajeureApr 3, 2024

@briankrebs It looks mostly like they introduced some minor bugs, but nothing like the xz thing.

Personally my guess is they were contributing to another project in the same space to give themselves cover and establish contributions, even if minor ones, and they just happened to be moderately buggy like a great many contributions are. But that's just a guess.

Show threadDavid BloomApr 3, 2024
@briankrebs my guess is they were testing the waters to see how much scrutiny the change got. the bug it introduces is not particularly severe on its own, and could be explained away as a careless mistake if anyone did flag it, so i think it was really more of a canary than anything else.
Show threadBig Kumera EnergyApr 3, 2024
@briankrebs certainly looks like this particular PR introduced a flaw in terminal control character handling in archives
Show threadBig Kumera EnergyApr 3, 2024
@briankrebs I swear if this is the university of Minnesota again, still trying to find ways to mess with foss >:[
Show threadVessOnSecurityApr 3, 2024

@briankrebs It does. But the vulnerability introduced here is nothing spectacular and could be just a honest mistake or carelessness.

Even if it was intentional, it was of rather limited scope - would affect only terminals that can interpret ANSI escape sequences (for instance, CMD.EXE can't do it unless you fiddle with it) and the victim would have to be convinced to process a booby-trapped archive.

Show threadNot SimonApr 3, 2024
@briankrebs there is a coordinated libarchive review effort tracked in https://github.com/libarchive/libarchive/issues/2103
re-review commits · Issue #2103 · libarchive/libarchive

In light of the xz backdoor (https://www.cve.org/CVERecord?id=CVE-2024-3094) it seems prudent to (at least) review again commits by the same author. I believe these are the associated commits: 0f74...

GitHub
Show threadBen AvelingApr 3, 2024
@briankrebs Long shot, but if someone was unsuccessfully trying to (ab)use untaring to put files somewhere they shouldn't, they might just have wanted better quality error messages.
Show threadabadideaApr 4, 2024
@briankrebs I am going to throw my hat into the ring for this being a smoke test for how much scrutiny random minor changes receive, because it’s clearly a regression even if it doesn’t necessarily introduce a new vuln, and is easily deniable as being an innocent mistake if someone does complain.
Show threadJan Wildeboer 😷Apr 3, 2024
@briankrebs I'll go out on a limb. Knowing how the access vectors this backdoor planned to use were being shut down, they sped up the process to not lose the bit of possibilities they thought to have. So the work became distributed, another member of the team was brought in. This panic reaction turned out to be fatal.
Show threadkkellerApr 3, 2024
@jwildeboer @briankrebs was anyone suspicious of the unusual commit times before the backdoor was announced?
Show threadJan Wildeboer 😷Apr 3, 2024
@kkeller @briankrebs Nobody looked at the time. Now we do.
Show threadJoe Cooper 🇺🇦 🍉Apr 4, 2024
@kkeller @jwildeboer @briankrebs what's "unusual" for a hobbyist working on a hobby project? My commit history is all kinds of bonkers, especially for my personal projects. I doubt anyone ever even thought about when commits were made until after we knew they were malicious.
Show threadasclepiadaeApr 4, 2024
@swelljoe @kkeller @jwildeboer @briankrebs unusual is that in this case the cluster of malicious commits were made outside of the other commits made over a two year period. That long a time period allows for a fairly clear view of “normal” behaviour for that individual, even working on “hobbyist” frequency.
Show threadCloud ManulApr 3, 2024
@briankrebs Have you heard anything about law enforcement involvement? I see lots of trouble coming up, starting with the question of which specific agency is leading since we have no definitive knowledge of the attacker's country of origin yet.
Show threadGeoff SimmonsApr 3, 2024
@briankrebs IIRC around the same time, sock puppets re-appeared to lobby for inclusion of the backdoored version in at least Debian unstable. I think @rotopenguin has pointed to the most plausible answer. The proposed PR for systemd meant that the window for the exploit would be limited. If they could get it into the next Debian release, and possibly the Ubuntu update in April, at least it would be in the wild for a few months. They were running out of time.
Show threadFrederikApr 3, 2024

@briankrebs A nation state actor might have one team who's only responsibility is innocent open source work and another team who is responsible for developing the backdoor and perhaps control of the account was handed over once the backdoor was ready? Perhaps the backdoor team was less careful and committed "in their own timezone"?

Or perhaps "grooming" of the account was outsourced to another group/company before the execution of the backdoor?

Show threadGlobaltomApr 3, 2024
@briankrebs Or were they just trying to hide them?
Show threadAcalarchApr 3, 2024

@briankrebs

I haven't had a chance to deep dive on this but it seems someone might have built and sold a legit persona (or someone handed it to another team)

Show threadBenAvelingApr 3, 2024
@briankrebs I'd suspect that 'Jia Tan' was physically based somewhere in the Americas and that someone else, possibly based in China, used the account to do the commits in the malicious cluster. And yes, I'm quite curious about the earlier 3am commit.
Show threadVessOnSecurityApr 3, 2024

@briankrebs If the account was hacked, Jia Tan would still be around, proclaiming loudly "hey, it wasn't me, honest". The fact that he has disappeared, means that he knew perfectly well what was happening.

The out-of-character commits probably reflect the time when he forgot to set the time zone to UTC+8 and remained on his "native" time zone of UTC+2.

And, yes, he *was* on a deadline. The systemd people were about to introduce a change that would have prevented the backdoor from working.

Show threadFarce MajeureApr 3, 2024
@bontchev @briankrebs and also they did it ~6 weeks before Fedora 40 and Ubuntu 24.04, which were their best chances of actually getting it out in the world.
Show threadwaldiApr 3, 2024
@bontchev @briankrebs The timestamps are all in UTC. You need to change the clock of your system, not the timezone, to get this behaviour. However you could also check the separate timestamps of the GitHub pushes and pull requests, which can’t be spoofed.
Show threadPhilApr 3, 2024
@waldi @bontchev @briankrebs git timestamps can be easily munged with a hook script, too. No need to modify the system time.
Show threadvarx/techApr 3, 2024
@bontchev @briankrebs Or Jia is going to come back from vacation to a hacked account and a *very* full inbox. 😆
Show threadṫẎℭỚ◎ᾔ ṫ◎ℳApr 3, 2024
@briankrebs Did you see this?  
https://www.bleepingcomputer.com/news/security/us-state-department-investigates-alleged-theft-of-government-data/
US State Department investigates alleged theft of government data

The U.S. Department of State is investigating claims of a cyber incident after a threat actor leaked documents allegedly stolen from a government contractor.

BleepingComputer
Show threadSpaceLifeFormApr 3, 2024

@briankrebs

Multiple people in different timezones is my guess.

Show threadTravis LankowApr 3, 2024
@briankrebs https://github.com/systemd/systemd/pull/31550 likely increased pressure to get the backdoor into production distros, as that change would have impacted its usefulness. There was another proposed change IIRC that would have done similarly but I can’t find it at the moment.
Dynamically load compression libraries by teknoraver · Pull Request #31550 · systemd/systemd

Dynamically load compression libraries (LZ4, ZSTD, LZMA) so we can reduce the size of the initram images by omitting libraries which aren't really used.

GitHub
Show threadheinrichApr 3, 2024
@briankrebs @briankrebs what it really means, is the controls were not intelligent enough to realize that there was a (lack of a) pattern. ZT & AI would have.
Show threadJesse SantangeloApr 4, 2024
@briankrebs Team effort. One guy writes a lot of normal code, someone else borrows the account to send in their specially crafted malicious code. But they were sloppy and didn't bother trying to sync their efforts up into the same timezone. That's my theory, at least.