The setup behind the CVE-2024-3094 supply-chain attack is fascinating. I originally wanted to finish and share a tool to audit other OSS projects for anomalous contributor behavior, but I feel what I found trying to MVP it is way more interesting. 🧵 1/25
25/25 It feels very lucky that it was discovered at the stage it was. I hope with this attack on people's minds, other OSS projects in similar positions consider doing tabletop scenario exercises for this kind of attack and how they can prevent/detect it. Thanks for reading!
23/25 I wonder how many other high-effort "fake identities" are still in the infiltration stage, building trust with maintainers of other quiet or older projects that are a valuable target for attackers but aren't necessarily understood as one.
24/25 If the injected code was more conservative selecting targets and didn't have a performance impact so significant that someone who (in their own words) "is not a security researcher/engineer" began to investigate, how long could this have gone undetected?
7/25 I started manually auditing the xz repo. Another surprise was reading the test file README in xz:
"Many of the files have been created by hand with a hex editor, thus there is no better "source code" than the files themselves."
With hindsight of the test file backdoor... 😅
16/25 From 2022 though a focus on xz-utils, even representing it in other projects! In Google's oss-fuzz, they disabled the same compiler feature their backdoor uses to intercept execution. And then changed the primary contact, so any bugs it did manage to find went to them...🤔
2/25 If you haven't, please read the full
@Openwall
mailing list disclosure. The first advisory summary a friend shared with me had such a high-level overview that I feel I initially grossly underestimated the level of sophistication of this attack.
15/25 The other contributor has authored 76% of commits, incl. the first. So between them, 95% of all commits.
But their GH account was created in 2021! Before working on xz, they ... tried to make libarchive auto-download combinations of dependencies that didn't make sense 🤔
5/25 Suddenly I had a lot of questions. Why did sshd/OpenSSH load xz-utils if OpenSSH doesn't depend on it? As I understand now, official OpenSSH does not but linux distro packages often patch it to support systemd, which does. (still not 100% - please correct me if I am wrong!)
12/25 But as I was running my MVP script in the xz-utils repo, I realized that if this user was a 'fake identity' as suspected, the creator had been anything but lazy. This is by far the most work/time/persistence I've seen go into an attack that anyone can follow chronologically
22/25 At the time of writing they are even still listed as co-maintainer on the sponsoring project's website too. My point isn't to goof on the project, but rather to highlight the level of trust and access they achieved while infiltrating the project.
8/25 When I looked for commits in other related projects adding new binary files, my first hit was a test fixture binary in zstd - also a compression lib too!
The same commit also had automation to regenerate and detect the file changing.
9/25 I don't think this is by any means the single/most important factor that lead to the attack, but I did want to show them in contrast to at least highlight that there is a better way of doing this, and that CI/test infra hygiene is worth continuously reviewing and bettering.
14/25 Since their first commit in Jan 2022, they have authored a total of 451 commits in xz-utils main banch. That's 19% of all main branch commits in just over 2 years. The project's first commit (when it migrated to git) was over 16 years ago!
11/25 I spent way too much time keeping it as a one-liner, but I now had something to find each binary file in a repo, the commit author who last modified it and agg. git stats, recursive-extract binwalk / strings it, and print an (ugly) plaintext report.
3/25 Hackers tend to be lazy. When I heard "fake identity", I was thinking automation of "grammar fix" OSS contributions on many fake identities, farming activity on projects, and only after the identity met a threshold would an attacker even assess it for repetitional value.
18/25 While that would prevent tools like binwalk from properly identifying the machine code it contained, they went so far as to make their scrambled backdoor test file have similar artifacts to others. Shown is a diff of the strings between the backdoor and another test file.
20/25 I wondered why risk adding 2 new test files that didn't even get used. The disclosure actually mentioned 5.6.0 and 5.6.1 being vulnerable with different payloads. This is how the original backdoor payloads were added... hiding in plain sight 🥲
17/25 I'll stop with all the 🤔 sorry 🙇♀️
Once they decided they were ready to launch their backdoor, they still checked every detail carefully. They injected their code by a mix of an unclear-but-uninteresting build script addition that descrambled the test files in the project.
21/25 The repo is currently unavailable, but in an earlier PR I found on web archive they were merging their own changes without review as early as 2023. I unfortunately couldn't get the PRs for the backdoors, but I wonder if that PR had any review at all
13/25 Factoring in the lack of any other online web presence, as of now I would be incredibly surprised to learn this account was not created by the backdoor commit author, most likely with xz as a target to try to infiltrate.
Regardless, they have had a very busy past few years!
19/25 Last one! When I was skimming the binwalk outputs, I thought I ID'd another backdoor payload when it found an an xz-compressed x86 binary with a different name. Turns out this has been there since 2009, with the context explained in the git commit message.
6/25 My thought then was to audit other projects for anomalous contributor behavior - especially ones that may have been an "unclear" dependency. But I was still confident the agg. stats of the backdoor commit author's git contributions would have patterns of automation too.
10/25 I'm now so emotionally invested I want to start script something. I iterated by auditing a few lower-level library projects, and adding new ideas as they came to me. I was also very eager to (and honestly, way, way too late...) start testing my script in the xz-utils repo!
4/25 It still seemed unlikely to fool project maintainers though. Even with newer technologies like ChatGPT, I thought this would need to be done on a scale that would leave some identifiable patterns in activity.. Then I started to read the full original disclosure.
I disagree with many of the "long-term, planned = state-sponsored attacker" takes on the
#xzbackdoor
.
I once spent a few months becoming a Facebook subcontractor, just to poke around in their internal SRT review tool for bugs.
(I explicitly asked their team first if I could try!)
did you know google analytics actually have an apt division that read your leaked password reset tokens in referrer header and hijack your users accounts
It's possible further analysis of the backdoor payload might make attributing it possible.
But I don't think creating a new identity, putting in work to build trust/reputation, and infiltrating a smaller OSS project is only something state-sponsored attackers are capable of.
I was mainly just curious and only found a couple IDORs in the end, but this isn't my only personal story like this. A lot of my researcher friends have many of their own. I wouldn't underestimate what an individual can do, especially if they had political/financial motivations.
I once reported a bug for a large airline where the example requests in the API docs for their frequent flyer program included had a valid “super admin” API key in the headers. Authentication for external APIs is really in need of a better (and still user-friendly) approach.
Apparently, this exploit happened because the gov developer wrote a tech blog on CSDN and accidentally included the credentials.
1 billion records of private citizens' data. 😭
TIL “In the United Kingdom, if a gingerbread man has just two chocolate eyes, it is legally a biscuit (tax exempt). However, if that gingerbread man is also wearing chocolate pants or a shirt, it is now legally taxed at a standard rate of 20%”
I watched Kiki's Delivery Service for the first time since I was a kid (like 8 or 9) over the weekend, and only now realised that it's message is about burnout...
@JetBrains
were including valid Apple code-signing and Artifactory credentials in Windows PyCharm builds for a few months, due to their build process dumping env vars in to one of the included stub files. Great response from their team - credentials were rotated within hours 1/2
i have like 100% feedback on 160+ transactions on merukari (a Japanese buy/sell app) and i still get anxiety every time i finalize a transaction on it. i think if someone gave me negative feedback i wouldn't be emotionally prepared to handle it.
thinking about the fact that there was a point in my life where the majority of my day was writing what should have been an actual server app in varnish configuration language with embedded ruby to get around things like no for loops. i do not miss that.
i made this in February and never shared it because I figured no one would find it funny except me but after seeing some similarly really specific memes im proud to share Joe Rogan and Ben Shapiro talking about how mongo is web scale
the year is 2047,
@nike
will only sell shoes in raffles with limited quantities. because of this, shoes are considered a scarce luxury for the upper class. everyone else wears fake yeezys.
@_wirepair
Yes but the building is sharing a 1GB fiber (this is to the IDF), need to see if I can negotiate with the management company about pulling my own cable in :)
@NathOnSecurity
@evanricafort
@Bitquark
actually many of my Japanese friends thought my primary income was from extortion until I showed them the explanation on bugbountyjp
@danielshi
@patio11
They own/operate the mall and built one of the tower mansion complexes there too (City Tower Ariake Garden). They're trying the same thing at Haneda Airport minus the tower mansion, but with an... onsen? 🤔
@katakana_kurisu
@Yetska
I actually started writing a blog post a few weeks ago about names in Japan and where a lot of these issues come from - really should finish it 😅
@TokyoFashion
Kitaya park too in Shibuya! I remember when it used to be an actual park, and now it’s a blue bottle coffee and a rotation of food trucks :/
@patio11
Please consider swapping APA Hotel out for another comparable hotel chain (JR Mets, Toyoko Inn, Dormy Inn etc). Their president has written a revisionist history book denying the Nanking Massacre and leaves copies of it in their rooms