They used Outlook Web App - runs the Exchange Server codebase btw - to craft tokens to bypass auth.
There's some clever wording in blog around only impacting OWA. OWA is a part of Microsoft 365 and Exchange Online.
The problem was discovered by the US Government and reported to Microsoft. https://edition.cnn.com/2023/07/12/politics/china-based-hackers-us-government-email-intl-hnk/index.html
This one looks like a huge mistake, a consumer MSA key (managed end to end by Microsoft - there's no external logs) was able to forge any Azure AD key.
It's only become public it appears as the US Government told Microsoft, which forces public disclosure.
CISA's advisory on the Microsoft 365 compromise is wayyyyyyyyyyy better than the Microsoft advisory - contains actionable hunting and logging information. Kinda nuts that the US Government are providing better information about Microsoft than Microsoft.
Okay - I found a victim org.
The situation for them is 😬
MS are going to have to release more info, methinks.. or I crank out the blog writing.
Really good Washington Post piece on the breach of Microsoft 365’s email service.
- hackers accessed customer emails for a month
- Microsoft didn’t notice
- USG had to tell them
- The access to generate tokens very likely came from MS being hacked and not realising
Talked to another impacted victim org in the Microsoft 365 hack, they basically got no actionable info from MS. Basically ‘lol you got hacked’ with wordsmithing and padding. 👀😬
I think I’m going to post hunting queries for this with an MS Paint logo.
🎶 regulation 🎶
I agree with CISA here (and have publicly for years) - security access logs for customers own services shouldn't be locked behind E5 per user licensing.
Yes, it will cost Microsoft money in upsell. They're more profitable than a large portion of the UK economy; they can afford it.
I should also point out the reason Microsoft was able to tell orgs specifically that they'd be targeted even when they didn't have E5 is MS already store the logs anyway.
“We don’t have any evidence that the actor exploited a 0day." say Microsoft. Their first blog on this says “exploit” - so are MS saying they don’t patch vulnerabilities in their cloud? 🤔
Their latest blog also says “This was made possible by a validation error in Microsoft code” - which is a vulnerability. Which is a 0day as it was under exploitation before Microsoft knew of it existing.
Microsoft lying to media and customers is not a good look.
All it took was Exchange Online in GCC and GCC High getting breached
Non-E5 users to get some security log availability finally.
More details about the Microsoft 365 Exchange Online breach in this article.
Although not stated, orgs are struggling to understand the scope of the breach due to audit log limits on MailItemsAccessed - it stops recording after 1k items. https://www.wsj.com/articles/u-s-ambassador-to-china-hacked-in-china-linked-spying-operation-f03de3e4
Just to loop this thread into this thread - I took a look at the attack path used in the M365 customer data breach.
A key part of the attack chain was documented by Microsoft at BlackHat in 2019.
Attached: 2 images Been looking at Microsoft 365 email breach some more - it looks like Microsoft were aware of issues in same token validation space in Exchange Online 4 years ago. MS did a talk at BlackHat about it, after somebody external pointed out an invalid token allowed any email box to be accessed via consumer Outlook.com. They fixed that issue - but still allowed any valid MS token to access any email, so the threat actor stole one of the MSA certs. Talk: https://www.youtube.com/watch?v=KN6e1mqcB9s
Wiz have an in-depth look at what they think happened at Microsoft over the Microsoft 365 breach.
They nail a new detail - one of the 'acquired' signing keys expired in 2021, but apparently it was still valid in Microsoft's cloud services. https://www.wiz.io/blog/storm-0558-compromised-microsoft-key-enables-authentication-of-countless-micr
Our investigation of the security incident disclosed by Microsoft and CISA and attributed to Chinese threat actor Storm-0558, found that this incident seems to have a broader scope than originally assumed. Organizations using Microsoft and Azure services should take steps to assess potential impact.
YOU MUST ONLY READ THE OFFICIAL BLOGS
there is no breach
there is no vulnerability
there are no zero days
*jedi wave*
https://therecord.media/microsoft-disputes-report-on-chinese-hacking
The Microsoft write up on how Microsoft 365 got owned to steal customer emails is out. It’s really good and honest from a technical level I think, if you’ve been following the details closely. Top points to the US Gov for forcing public disclosure originally btw.
There’s a pretty good look at unanswered questions the MSRC blog on the Microsoft 365 customer data breach in this: https://arstechnica.com/security/2023/09/hack-of-a-microsoft-corporate-account-led-to-azure-breach-by-chinese-hackers/
Unsurprisingly MS aren’t using words like ‘breach’, ‘vulnerability’ etc when clearly it was both. It’s almost like there’s misaligned incentives.
Other obvious issues include a compromise in 2021 where the threat actor took process dumps etc but nobody checked what they were doing (you live and learn etc), no HSMs etc. Assume MS are compromised.
This TechCrunch piece has one extra detail not in the MSFT blog on the Microsoft 365 data breach - access was gained via session token theft.
To expand, Microsoft use Azure AD MFA, which has a problem with session token theft. https://techcrunch.com/2023/09/08/microsoft-hacker-china-government-storm-0558/
US State Department have gone on the record about how they found the Microsoft 365 data breach.
They set up a detection rule called Big Yellow Taxi two years ago to look for unknown AppIDs in OfficeActivity, which ultimately saved Microsoft’s ass.
https://www.politico.com/news/2023/09/15/digital-tripwire-helped-state-uncover-chinese-hack-00115973
Chinese hackers who breached Microsoft's <a href="https://www.reuters.com/markets/companies/MSFT.O" target="_blank">(MSFT.O)</a> email platform this year managed to steal tens of thousands of emails from U.S. State Department accounts, a Senate staffer told Reuters on Wednesday.
Microsoft have announced they are going to start using Azure HSM for their own services finally, after being cyber bullied by GossiTheDog. https://www.microsoft.com/en-us/security/blog/2023/11/02/announcing-microsoft-secure-future-initiative-to-advance-security-engineering/
(It’s actually a really good blog with a bunch of good ideas, if you ignore the AI stuff).
Absolutely blistering independent review into Microsoft 365 breach early last year is due this week from Cyber Safety Review Board, highlights huge problems with Microsoft’s security.
I did not participate.
Contains something I didn’t know - last month, Microsoft quietly corrected a blog to say they never found the crash dump with the certificate, so do not know how China got it. They did not store it in a HSM.
References earlier breach they hadn’t disclosed.
Report into MS breach is out: https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf
I had a tweet in 2021 saying MSTIC should not use the Nation State Notification process to hide breaches from the public.
That was a reference to the Affirmed Networks breach - aka Azure for Operators - listed in this report. They hid it.
The website for Azure for Operators at the time had Satya’s face on it.. that breach, which they refused to share details about, apparently led to this one.
I’ll save full thoughts for later as I need to digest the report, but I will say to Microsoft’s credit, I’ve heard they got the memo on security and plan a range of things including org and governance changes.
IMHO MS need a properly centralised security op model, like you see at.. well.. every other org. And then robust control implementation, lead by risk, blanketed everywhere.
Security should be treated like safety - if you endanger customers, you on the naughty step.
Digging through my old tweets - this one from 2022 was after finding out Affirmed Networks aka Azure for Operators had been breached by STORM-0558 (China).
You will not know about the breach, as it isn't recorded anywhere online other than this tweet. From what I can gather they also failed to tell the US Government about it.
The German security services are suing Microsoft over failure to disclose information about one of the Microsoft 365 security breaches: https://www.heise.de/en/news/BSI-verklagt-Microsoft-auf-Herausgabe-von-Informationen-zu-Security-Desaster-9722507.html
I doubt they will get very far as Microsoft takes steps to avoid legal disclosure in security incidents.
@GossiTheDog ISTR hearing that in 2003.
(Not to snark when they're trying their best and have come a very long way since the Gates memo, but I am weak...)
@BibbleCo @GossiTheDog There was a definite cycle from 2003-2014 (when the TwC org was disbanded). In my experience, that was the high water mark for security at Microsoft -- after that, I felt it became much easier for internal orgs to ignore security for cost, convenience, or a desire to deliver ads on the desktop.
The last major argument that I had at Microsoft, in 2017, was about audit logging in O365. It was disabled by default which led to quite a few companies getting as far as finding a security incident and, when their IR teams went to look for audit logs, coming up blank. The O365 org had the audit logs anyway but would refuse to retrieve them for customers which is some dramatic anti-customer bullshit. I managed to get logs for a number of customers solely because they paid Microsoft for my team's help in investigating their incident which is, with the clarity of hindsight, monetizing anti-customer bullshit.
That's not the argument I was talking about, though. I made the case to O365's CISO that audit logging should be on by default and, while he agreed in principle, he indicated that it would cost about $4mil/month in storage costs. He still agreed to try to make it happen and, after I left Microsoft, they did turn audit logging on by default. I felt pretty accomplished about being part of that.
...only to discover, last year, that they simultaneously locked the most useful audit logging behind E5 licenses, leading to a situation where many customers couldn't even figure out if they'd been compromised.
@ancatdubh @BibbleCo @GossiTheDog Well, I, too, am hopeful that being chastised by CISA in 2024 will have some of the same effect on Microsoft that being chastised by Bill Gates did in 2002.
And, in fairness, the problem space has moved a lot in 22 years. In 2002, we were talking purely about product problems -- Windows, Internet Explorer, Office. The solutions started with an embrace of the SDLC and radiated outward from there.
Those problems will never be fully solved and always require vigilance but, on the product front, I think Microsoft does as good a job as anybody in the business. I think Windows is probably more secure, for example, than comparable OSes. But, we've largely moved from software-as-a-product to software-as-a-service and organizations have to wrestle with the security of how they operate and, if they're a service provider, of how their customers operate. I've long joked-not-joked that we need a Secure Operations Lifecycle to go with the Software Development Lifecycle.
@neilcar @ancatdubh @GossiTheDog Very hard to normalise & quantify "secureness" across platforms, even if just considering the base OS, though I've wasted many hours over the decades reading about people trying to do so.
Personally, I continue to use an incredibly obsolete EOL'd system, relying on obscurity, being a low value target, and luck. When I retire from my current occupation (looking after my aging parents), I'll find time to establish a proper home office / labby type environment and get back to the security update fandango, streaming logging to something appropriate, etc. W2K forever! ;)
@neilcar @GossiTheDog "...audit logging in O365 [..] was disabled by default" --
Happily, I never worked at an O365 customer, so didn't know that. As you say, pretty evil. Had I, it wouldn't have violated the law of least astonishment for me... Proper logging should always by on by default, and if the $4m was that big of a deal for MS, they should have rolled it into the basic product cost and spread it across the customer base. Very poor.
@BibbleCo @GossiTheDog Weirdly, I think it was more banal than evil but the difference is often in how we perceive the actor than in the action itself.
And, the sad truth is that O365 is, simultaneously, not great AND the best available hosted e-mail/productivity suite in the market. Maybe Google will apply some Mandiant-sauce to Google Apps but I really wouldn't want to have to manage, detect, and respond in that platform for any large org.
@neilcar @GossiTheDog Concur.
And didn't someone or other once have something to say about banality and evil?
(Godwin? Never heard of it ;) )
@neilcar @BibbleCo @GossiTheDog I'd rather fuck my own asshole with a chainsaw dipped in salt and broken glass than handle an incident in Google environments.
It's laughably pathetic how shit that whole ecosystem is.
@adrianco @jawnsy @GossiTheDog even EVP can't cut it if you're swimming upstream against a long established culture.
I've seen CEOs who've swapped out pretty much the entire exec team, and still been unable to make any headway against a 'deep state' praxis (usually centred around finance, HR and legal).
@cstross is right about corps being slow AIs (and people being the gut flora). You can change an awful lot and the stink's still the same.
@danhon @adrianco @jawnsy @GossiTheDog I can't think of a single case study where that's been successfully done.
What can work is the equivalent of a brain transplant - merge with another corp and explicitly choose its (better) systems and culture.
Though that can go awry too - viz Boeing post MacDonnell Douglas.
@adrianco @danhon @jawnsy @GossiTheDog
Indeed. Though I feel like that's a surface culture, and there's a deep culture beneath that's likely to remain unchanged: what does it take to buy something, or do travel, or spend $ on storing logs, or make the extra effort to add an OpenSSF Scorecard to a repo, (or use you own HSM service)?
There were plenty of IBM CEOs who bragged about changing the culture there, and were lauded in the biz press. But in the long span of history, did they? Really?
@SteveSyfuhs @adrianco for the record I agree here. At Charlie’s level, he needs to look at strategy - which takes years to turn around. Rightly so. All the signs are he’s doing a great job I think, because wheels are starting to turn.
Also some cultural change, eg ‘bring out your dead… before attackers do’. There’s lots of very smart people at MS who know about all these problems individually, but organisationally they haven’t been incentivised to say it and fix it IMHO.
@adrianco
In my experience of large enterprise cloud sales on the buying side, Deep Dives, careful RFPs & technical due dilligence by experienced practitioners showed Azure to be security scotch tape and bubblegum, but they would win anyway because relationships, pricing, bundling and because it's Microsoft.
IBM disease, v2.
Also MS was very willing to play race to the bottom on price to get the large enterprise deals I was involved with.
@GossiTheDog @SteveSyfuhs
@adrianco
It's still amazing to me how little execs weight these kind of future & reputational risks. When (not if) the risk realizes, they inevitably are material.
But people focus on the odds, not the stakes, and they rarely pay attention to cumulative risk over time.
@GossiTheDog @SteveSyfuhs
A 10% of global annual revenue fine for each breach or security issue might also get the message across more directly.
Perhaps the shareholders won't put up with it if their stock dividends are directly affected 🫤🤷♂️
@GossiTheDog “Throughout this review, the board identified a series of Microsoft operational and strategic decisions that collectively points to a corporate culture that deprioritized both enterprise security investments and rigorous risk management,”
...all while simultaneously blaming customers who fall foul to hacks for failing to "properly secure" their not-secure-by-default products
@GossiTheDog Wait what!? The very cool crash dump theory (though it should have been prevented) was just a made-up guess or something!?
Could Microsoft not afford to pay Microsoft for longer retention on the crash dump servers?
sweet jeebus.... 😬