Kevin BeaumontJul 12, 2023
Microsoft quietly snuck out a blog yesterday to say that Office 365 got compromised by China and used to steal emails. Thread follows. https://msrc.microsoft.com/blog/2023/07/microsoft-mitigates-china-based-threat-actor-storm-0558-targeting-of-customer-email/
Microsoft mitigates China-based threat actor Storm-0558 targeting of customer email | MSRC Blog | Microsoft Security Response Center

Microsoft mitigates China-based threat actor Storm-0558 targeting of customer email

Show threadKevin BeaumontJul 12, 2023

They used Outlook Web App - runs the Exchange Server codebase btw - to craft tokens to bypass auth.

There's some clever wording in blog around only impacting OWA. OWA is a part of Microsoft 365 and Exchange Online.

The problem was discovered by the US Government and reported to Microsoft. https://edition.cnn.com/2023/07/12/politics/china-based-hackers-us-government-email-intl-hnk/index.html

Show threadKevin BeaumontJul 12, 2023
Microsoft have not linked the blog on @msftsecintel or @msftsecresponse Twitter accounts or social media, instead linking pieces yesterday about an unrelated phishing campaign.
Show threadKevin BeaumontJul 12, 2023

This one looks like a huge mistake, a consumer MSA key (managed end to end by Microsoft - there's no external logs) was able to forge any Azure AD key.

It's only become public it appears as the US Government told Microsoft, which forces public disclosure.

Show threadKevin BeaumontJul 12, 2023
Although MS haven't called this a vulnerability, haven't issued a CVE or used the term zero day.. they don't issue CVEs for cloud services, forging a token is a vulnerability, so it's a zero day.
Show threadKevin BeaumontJul 12, 2023

CISA's advisory on the Microsoft 365 compromise is wayyyyyyyyyyy better than the Microsoft advisory - contains actionable hunting and logging information. Kinda nuts that the US Government are providing better information about Microsoft than Microsoft.

https://www.cisa.gov/sites/default/files/2023-07/aa23-193a_joint_csa_enhanced_monitoring_to_detect_apt_activity_targeting_outlook_online.pdf

Show threadKevin BeaumontJul 12, 2023
Another element - to spot this activity, the US government used enhanced logging aka Purview Audit (Premium) logging - the US government had a huge public fight with Microsoft over this a few years ago over cost, to get access. Turns out they needed it indeed.
Show threadKevin BeaumontJul 12, 2023
Does anybody have the AppID used in the Microsoft 365 compromise? -> [email protected]
Show threadKevin BeaumontJul 12, 2023
WSJ reporting the Microsoft 365 hack was used to spy on the State Department. https://www.wsj.com/articles/chinese-hackers-spied-on-state-department-13a09f03
Chinese Hackers Breached Email of Commerce Secretary Gina Raimondo and State Department Officials

Hackers didn’t appear to gain access to national security information

WSJ
Show threadKevin BeaumontJul 12, 2023
For anybody interested - the β€œacquired Microsoft account (MSA) consumer signing key” used in this must have come from inside Microsoft’s internal network.
Show threadKevin BeaumontJul 12, 2023
The teams who worked on the Microsoft 365 breach of customer data are having a snow day still, I see.
Show threadKevin BeaumontJul 12, 2023

Okay - I found a victim org.

The situation for them is 😬

MS are going to have to release more info, methinks.. or I crank out the blog writing.

Show threadKevin BeaumontJul 12, 2023

Really good Washington Post piece on the breach of Microsoft 365’s email service.

- hackers accessed customer emails for a month
- Microsoft didn’t notice
- USG had to tell them
- The access to generate tokens very likely came from MS being hacked and not realising

https://archive.is/2023.07.12-230927/https://www.washingtonpost.com/national-security/2023/07/12/microsoft-hack-china/

Show threadKevin BeaumontJul 13, 2023
None of these would have helped, since the breach was at Microsoft’s end.
Show threadKevin BeaumontJul 13, 2023

Talked to another impacted victim org in the Microsoft 365 hack, they basically got no actionable info from MS. Basically β€˜lol you got hacked’ with wordsmithing and padding. πŸ‘€πŸ˜¬

I think I’m going to post hunting queries for this with an MS Paint logo.

Show threadKevin BeaumontJul 13, 2023

🎢 regulation 🎢

I agree with CISA here (and have publicly for years) - security access logs for customers own services shouldn't be locked behind E5 per user licensing.

Yes, it will cost Microsoft money in upsell. They're more profitable than a large portion of the UK economy; they can afford it.

I should also point out the reason Microsoft was able to tell orgs specifically that they'd be targeted even when they didn't have E5 is MS already store the logs anyway.

https://archive.ph/MFnxP

Show threadKevin BeaumontJul 14, 2023
On how the USG, European govs and Microsoft have been threat hunting the MS 365 breach, per Microsoft documentation on the logs... "If a mailbox is throttled, you can probably assume there was MailItemsAccessed activity that wasn't recorded in the audit logs."
Show threadKevin BeaumontJul 14, 2023
Really good new MS blog on the MS compromise - contains IOCs etc. I'll put MSPaint.exe down. https://www.microsoft.com/en-us/security/blog/2023/07/14/analysis-of-storm-0558-techniques-for-unauthorized-email-access/
Analysis of Storm-0558 techniques for unauthorized email access | Microsoft Security Blog

Analysis of the techniques used by the threat actor tracked as Storm-0558 for obtaining unauthorized access to email data, tools, and unique infrastructure characteristics. 

Microsoft Security Blog
Show threadKevin BeaumontJul 14, 2023

β€œWe don’t have any evidence that the actor exploited a 0day." say Microsoft. Their first blog on this says β€œexploit” - so are MS saying they don’t patch vulnerabilities in their cloud? πŸ€”

Their latest blog also says β€œThis was made possible by a validation error in Microsoft code” - which is a vulnerability. Which is a 0day as it was under exploitation before Microsoft knew of it existing.

Microsoft lying to media and customers is not a good look.

https://arstechnica.com/security/2023/07/microsoft-takes-pains-to-obscure-role-in-0-days-that-caused-email-breach/

Microsoft takes pains to obscure role in 0-days that caused email breach

Critics also decry Microsoft's "pay-to-play" monitoring that detected intrusions.

Ars Technica
Show threadKevin BeaumontJul 15, 2023
Show threadKevin BeaumontJul 19, 2023

All it took was Exchange Online in GCC and GCC High getting breached

Non-E5 users to get some security log availability finally.

https://www.wsj.com/articles/microsoft-to-offer-some-cybersecurity-tools-free-after-suspected-china-hack-6db94221

WSJ News Exclusive | Microsoft to Offer Some Cybersecurity Tools Free After Suspected China Hack

Company says it will make security logs available to customers with lower-cost cloud services

WSJ
Show threadKevin BeaumontJul 21, 2023

More details about the Microsoft 365 Exchange Online breach in this article.

Although not stated, orgs are struggling to understand the scope of the breach due to audit log limits on MailItemsAccessed - it stops recording after 1k items. https://www.wsj.com/articles/u-s-ambassador-to-china-hacked-in-china-linked-spying-operation-f03de3e4

U.S. Ambassador to China Hacked in China-Linked Spying Operation

Spying campaign also compromised State Department official who oversees East Asia

The Wall Street Journal
Show threadKevin BeaumontJul 21, 2023

Just to loop this thread into this thread - I took a look at the attack path used in the M365 customer data breach.

A key part of the attack chain was documented by Microsoft at BlackHat in 2019.

https://cyberplace.social/@GossiTheDog/110736594147931759

Kevin Beaumont (@[email protected])

Attached: 2 images Been looking at Microsoft 365 email breach some more - it looks like Microsoft were aware of issues in same token validation space in Exchange Online 4 years ago. MS did a talk at BlackHat about it, after somebody external pointed out an invalid token allowed any email box to be accessed via consumer Outlook.com. They fixed that issue - but still allowed any valid MS token to access any email, so the threat actor stole one of the MSA certs. Talk: https://www.youtube.com/watch?v=KN6e1mqcB9s

Cyberplace
Show threadKevin BeaumontJul 21, 2023

Wiz have an in-depth look at what they think happened at Microsoft over the Microsoft 365 breach.

They nail a new detail - one of the 'acquired' signing keys expired in 2021, but apparently it was still valid in Microsoft's cloud services. https://www.wiz.io/blog/storm-0558-compromised-microsoft-key-enables-authentication-of-countless-micr

Compromised Microsoft Key: More Impactful Than We Thought | Wiz Blog

Our investigation of the security incident disclosed by Microsoft and CISA and attributed to Chinese threat actor Storm-0558, found that this incident seems to have a broader scope than originally assumed. Organizations using Microsoft and Azure services should take steps to assess potential impact.

wiz.io
Show threadKevin Beaumont

YOU MUST ONLY READ THE OFFICIAL BLOGS

there is no breach
there is no vulnerability
there are no zero days
*jedi wave*

https://therecord.media/microsoft-disputes-report-on-chinese-hacking

Microsoft disputes report that Chinese hackers could have accessed suite of programs

Microsoft is disputing a new report that claims hackers may have had access to more parts of victims’ systems than previously known in a campaign that targeted dozens of organizations, including government agencies.

Jul 22, 2023 at 8:32amWeb
Show threadKevin BeaumontSep 6, 2023

The Microsoft write up on how Microsoft 365 got owned to steal customer emails is out. It’s really good and honest from a technical level I think, if you’ve been following the details closely. Top points to the US Gov for forcing public disclosure originally btw.

https://msrc.microsoft.com/blog/2023/09/results-of-major-technical-investigations-for-storm-0558-key-acquisition/

Results of Major Technical Investigations for Storm-0558 Key Acquisition | MSRC Blog | Microsoft Security Response Center

Results of Major Technical Investigations for Storm-0558 Key Acquisition

Show threadKevin BeaumontSep 6, 2023
One big thing missing from Microsoft’s blog (that was in the Wiz blog, and is accurate) - the MSA key expired in 2021. They weren’t checking the validity dates, either - customers might want to ask them if they fixed this.
Show threadKevin BeaumontSep 7, 2023
One extra thing to highlight - Microsoft’s blog doesn’t mention it, but they demo’d the technique of using a signing key to access email from a different account using M365 on stage at BlackHat 3 years ago and made various recommendations to stop it happening again... which weren’t implemented. https://www.youtube.com/watch?v=KN6e1mqcB9s
Preventing Authentication Bypass: A Tale of Two Researchers

YouTube
Show threadKevin BeaumontSep 7, 2023

There’s a pretty good look at unanswered questions the MSRC blog on the Microsoft 365 customer data breach in this: https://arstechnica.com/security/2023/09/hack-of-a-microsoft-corporate-account-led-to-azure-breach-by-chinese-hackers/

Unsurprisingly MS aren’t using words like β€˜breach’, β€˜vulnerability’ etc when clearly it was both. It’s almost like there’s misaligned incentives.

Other obvious issues include a compromise in 2021 where the threat actor took process dumps etc but nobody checked what they were doing (you live and learn etc), no HSMs etc. Assume MS are compromised.

Microsoft finally explains cause of Azure breach: An engineer’s account was hacked

Other failures along the way included a signing key improperly appearing in a crash dump.

Ars Technica
Show threadKevin BeaumontSep 8, 2023

This TechCrunch piece has one extra detail not in the MSFT blog on the Microsoft 365 data breach - access was gained via session token theft.

To expand, Microsoft use Azure AD MFA, which has a problem with session token theft. https://techcrunch.com/2023/09/08/microsoft-hacker-china-government-storm-0558/

TechCrunch is part of the Yahoo family of brands

Show threadKevin BeaumontSep 23, 2023

US State Department have gone on the record about how they found the Microsoft 365 data breach.

They set up a detection rule called Big Yellow Taxi two years ago to look for unknown AppIDs in OfficeActivity, which ultimately saved Microsoft’s ass.

https://www.politico.com/news/2023/09/15/digital-tripwire-helped-state-uncover-chinese-hack-00115973

How the State Dept discovered that Chinese hackers were reading its emails

The State Department relied on a clever alert system to uncover and unravel an advanced Chinese spying campaign that involved breaches of officials’ emails.

POLITICO
Show threadKevin BeaumontSep 27, 2023
60k emails of the US State Department were stolen from Microsoft 365 in this security breach. https://www.reuters.com/world/us/chinese-hackers-stole-60000-emails-us-state-department-microsoft-hack-senate-2023-09-27/
Chinese hackers stole emails from US State Dept in Microsoft breach, Senate staffer says

Chinese hackers who breached Microsoft's <a href="https://www.reuters.com/markets/companies/MSFT.O" target="_blank">(MSFT.O)</a> email platform this year managed to steal tens of thousands of emails from U.S. State Department accounts, a Senate staffer told Reuters on Wednesday.

Reuters
Show threadKevin BeaumontNov 2, 2023

Microsoft have announced they are going to start using Azure HSM for their own services finally, after being cyber bullied by GossiTheDog. https://www.microsoft.com/en-us/security/blog/2023/11/02/announcing-microsoft-secure-future-initiative-to-advance-security-engineering/

(It’s actually a really good blog with a bunch of good ideas, if you ignore the AI stuff).

Announcing Microsoft Secure Future Initiative to advance security engineering | Microsoft Security Blog

Read more about the objectives and strategy behind the new Microsoft Secure Future Initiative.

Microsoft Security Blog
Show threadKevin BeaumontApr 2, 2024

Absolutely blistering independent review into Microsoft 365 breach early last year is due this week from Cyber Safety Review Board, highlights huge problems with Microsoft’s security.

I did not participate.

Contains something I didn’t know - last month, Microsoft quietly corrected a blog to say they never found the crash dump with the certificate, so do not know how China got it. They did not store it in a HSM.

References earlier breach they hadn’t disclosed.

https://wapo.st/4cJpKtW

Microsoft faulted for β€˜cascade’ of failures in Chinese hack

The independent Cyber Safety Review Board’s forthcoming report knocks the tech giant for shoddy cybersecurity, lax corporate culture and a deliberate lack of transparency.

The Washington Post
Show threadrefreshingapathy Nov 2, 2023
@GossiTheDog how it should have been architected in the first place.
Show threadJay Thoden van Velzen β˜οΈβ€‹πŸ›‘οΈβ€‹Nov 2, 2023

@GossiTheDog

sweet jeebus.... πŸ˜¬β€‹

Show threadTanawtsNov 2, 2023
@GossiTheDog it's been long over due heh
Show threadsigi714Sep 28, 2023
@GossiTheDog So, just my inbox? ;)
Show threadDrew WSep 23, 2023
@GossiTheDog where did you learn the part about OfficeActivity, and do you have any reference kql?
Show threadAlesandro Ortiz πŸ‡΅πŸ‡·πŸ³οΈβ€πŸŒˆSep 23, 2023
@GossiTheDog The employees who worked on that specific rule must be so proud after they learned about the impact of these Microsoft vulnerabilities.
Show threadMike JπŸ‘ΉπŸ€ 🀘🏻Sep 23, 2023
@GossiTheDog You don't know what you got til it's gone, or you write a detection rule for it.
Show threadnakSep 23, 2023
@GossiTheDog capital T Truth right here: β€œIn an odd way, despite all the advances we’ve had in cybersecurity … it sometimes comes down to one person seeing something that’s anomalous,” Painter said.
Show threadDFIR JustinSep 23, 2023
@GossiTheDog Beautifully simple query and effective, love to see it. Was probably originally designed to catch shady O365 addons/Azure AD app registrations. Ended up detecting the PRC.
Show threadAmitai CohenSep 24, 2023
@GossiTheDog any ideas as to what "fire in a cluster, a volume" might mean here? (quoted from the article as the name of the alert) - maybe a cluster of email messages and a high volume of message downloads? or does "fire" just mean the alert triggered?
Show threadCJSep 8, 2023
@GossiTheDog But Kevin, the MS docs say token theft is thought to be very rare! Best not to worry about it and look the other way.
Show threadJames Guthrie 🏴󠁧󠁒󠁳󠁣󠁴󠁿Sep 8, 2023
@GossiTheDog wait, so TokenTactics or similar? They fell for a devicelogin prompt?
Show threadHacker MemesSep 8, 2023
@GossiTheDog
Show threadMike (no not that one)Sep 8, 2023
@i0null @GossiTheDog Nothing like multiple locks on a door made of paper mache
Show threadpetur πŸ˜ΆπŸ‡ΊπŸ‡¦πŸ‡΅πŸ‡ΈπŸ‡ΉπŸ‡ΌSep 7, 2023
@GossiTheDog that's quite a supply chain issue
Show threadRatanasecSep 7, 2023
@GossiTheDog That's some pretty serious skills and tradecraft to pull something like that off without MS noticing.
Show threadNeil Carpenter Sep 7, 2023

@Ratanasec @GossiTheDog I would argue that it's not really serious tradecraft -- I'm reading between the lines but it sounds like the compromise was limited to a single engineer's account. They don't go into details but this may have included access to a laptop/desktop ("Ooooo, memory dumps! Let's copy those somewhere") or it might've been e-mail/O365, which may have had sufficient data to access whatever file transfer/sharing platform was being used. Heck, the memory dump might've been stored in OneDrive.

And finding something like this in a memory dump and extracting it isn't particularly difficult, either.

But...my takeaway is that tradecraft usually comes into play in moving laterally and staying resident. Compromises of a single user where the attacker doesn't move around and doesn't stick around often don't create enough noise to get noticed & investigated (or taken very seriously).

Show threadsigi714Sep 7, 2023
@Ratanasec @GossiTheDog You can only notice whey you monitor.
Show threadNeil Carpenter Sep 7, 2023

@GossiTheDog A friend said something to the effect of "Nothing in the MSRC blog covers hardening the corporate network,I guess that's implied."

My response was:

'I think it's more "We fully expect corpnet to get compromised again and we're not sure we can prevent it, we're going to try harder to keep sensitive info off it."'

Show threadFrank BajakSep 7, 2023
@GossiTheDog And if you’re a Yank assume Msft is compromised. ;-)
Show threadhuynhjlSep 7, 2023
@GossiTheDog they don't provide reasons why they think the particular account being breached would be the one accessing the crash dump. "This account had access to the debugging environment". I wonder what type of hints they have that it is the responsible account as opposed to say an insider or some other compromised accounts they don't know about.
Show threadπŸ‡ΊπŸ‡¦ haxadecimal πŸš«πŸ‘‘Sep 6, 2023
@GossiTheDog
The new NSA key in Windows is much better hidden.
Show threadRTSep 6, 2023

@GossiTheDog

What was the timeline provided for when the compromise actually occurred?

Show threadBrianKrebsSep 7, 2023
@GossiTheDog Once again, the lack of logs proves to be Microsoft's undoing. Why do they ever have to delete their own logs? They're a cloud provider right?
Show threadDustin [BusySignal-KE2EFX]Sep 7, 2023

@briankrebs @GossiTheDog

heh...

I joked about how they couldnt even afford to store all the logs needed and I have Larry Literals in my DMs and mentions telling me how they cant keep everything due to possible regulations and all...

Like...
Total
LoL.

uhg

Show threadAMSSep 7, 2023
@briankrebs @GossiTheDog I'm sure many lawyer-careertimes were spent on determining the minimum log retention time that would not leave them criminally negligent.
Show threadSami JuvonenSep 7, 2023
@briankrebs @GossiTheDog I’ve been ordered to delete old logs as a systems guy (at multiple cloud providers) when lawyers got wind how much we were retaining.
Show threadRobbie Coleman Sep 7, 2023
@briankrebs
@GossiTheDog
Maybe it's s3 all the way down
Show threadhatmatterSep 9, 2023
@erraggy @briankrebs @GossiTheDog I mean, S3 logs are stored in S3 so I think you’re right
Show threadDoesSec πŸ” πŸͺͺ β˜• Sep 7, 2023

@briankrebs @GossiTheDog

The Storm-0588 hack reminds me of the movie Groundhog Day.

https://www.wired.com/story/the-full-story-of-the-stunning-rsa-hack-can-finally-be-told/

Show threadPerry LorierSep 7, 2023
@briankrebs @GossiTheDog I know at work we are forced to expire logs that might contain any user data due to GPDR.
Show threadDiego Elio PettenΓ²Sep 7, 2023

@isomer @briankrebs @GossiTheDog i don't like the phasing "due to GDPR" since retention is a nuanced and complicated matter.

But I can't get into too much of that for obvious reasons.

Show threadafxSep 7, 2023
@isomer @briankrebs @GossiTheDog So what are your retention periods then?
Show threadNicholas WeaverSep 7, 2023
@briankrebs @GossiTheDog
Random cynical thought:
MS dogfoods their own cloud. And since the cloud charges to actually keep the logs, they don't want to pay the internal recharge rate so they don't keep the logs.
Show threadTHIS ACCOUNT HAS MOVEDSep 7, 2023
@ncweaver @briankrebs @GossiTheDog lol
Show threadClaus Cramon HoumannSep 7, 2023
@GossiTheDog oh customers are and did :)
Show threadDan GoodinSep 6, 2023

@GossiTheDog Kevin, do you think Microsoft should disclose how the corporate account of its engineer got hacked? This seems like an important detail assuming Microsoft's goal is transparency.

Also, what do you think of the repeated use of the word "issue" in the post? Isn't "vulnerability" the correct term here?

Show threadJPSep 6, 2023
@dangoodin @GossiTheDog yes, I want to know how they just happened to have access when this crashdump just happened to come across the fence. Smells like persistent access to me. For how long? What else did they do or see? What other systems inside MS are at risk as a result?
Show threadi.grokSep 7, 2023
@daedalus @dangoodin @GossiTheDog if the crash dump had been exposed for years, they wouldn't need to lie in wait
Show threadAlesandro Ortiz πŸ‡΅πŸ‡·πŸ³οΈβ€πŸŒˆSep 6, 2023
@GossiTheDog Since this was detected by an enterprise customer, I wonder if the key leak would have been detected by Microsoft if the attacker had only used it to access consumer accounts (as intended). That would have been quite bad by itself.
Show threadDoesSec πŸ” πŸͺͺ β˜• Sep 6, 2023

@GossiTheDog

So the attackers discovered key material in Microsoft's super secure corporate environment that Microsoft itself didn't know had left the production environment, were able to exfiltrate the key material, and Microsoft doesn't want to have noticed any of this?

That sounds very unbelievable to me.

Show threadDave from accountingSep 6, 2023

@doessec @GossiTheDog
My mind was like "Riiiiight 🀫" APT just so happen to compromise an account and miraculously happen to find the proverbial needle in the haystack. And of course the log retention deleted all traces of this.

If I were to write the movie script, I would make it more believable, and put an insider threat in place that, when this person found that key just thought πŸ’°πŸ’°πŸ’° jackpot, retirement secured 😎

But, these guys would most likely have had that account compromised a long time and sifted through dumps on a regular basis looking for exactly something like this.

I would... πŸ€—

Show threadJaxxAISep 6, 2023
@GossiTheDog soooo... they're using 2 keys for everything and they don't rotate them? Eh?
Show threadSteffen ChristensenSep 6, 2023
@GossiTheDog It's a masterwork in incident reporting.
Show threadPreston de Guise πŸ³οΈβ€πŸŒˆβœπŸΌJul 22, 2023
@GossiTheDog β€œDo not look at the man behind the curtain!"
Show threadC-richJul 22, 2023
@GossiTheDog
Smilyanets and Cimpanu both always provide intriguing content
Show threadChristoph StoettnerJul 22, 2023
https://de.web.img3.acsta.net/r_1024_576/medias/nmedia/18/66/75/86/18953144.jpg only faith is required @GossiTheDog
Show threadSimonoidJul 22, 2023
@GossiTheDog
Show threadπ–‹π–‘π–”π–—π–Žπ–“ πŸ‡¨πŸ‡¦Jul 22, 2023
@GossiTheDog Don’t look up!
Show threadTim HergertJul 22, 2023
@GossiTheDog New MS Press lead
Show threadSam VargheseJul 22, 2023
@GossiTheDog https://itwire.com/open-sauce/azure-breach-microsoft-okays-wiz-post-on-continued-danger,-then-denies-it.html
iTWire - Azure breach: Microsoft okays Wiz post on continued danger, then denies it

Microsoft is continuing to obfuscate about a recent attack on its Azure cloud infrastructure, saying a post, that claims danger from the attack still exists, is speculative and not evidence-based. The company is mentioned in that same post as having checked the content for technical accuracy. Shir T...