Very big cyber incident playing out at Snowflake, who describe themselves as “AI Data Cloud”. They have a free trial where anybody can sign up and upload data… and they have.

Threat actors have been scraping customer data using a tool called rapeflake, for about a month.

The tl;dr of the Snowflake thing is mass scraping has been happening, but nobody noticed.. and they're pointing at customers for having poor credentials. It appears a lot of data has gone walkies from a bunch of orgs.

Snowflake is a big AI data company with a conference in the US next week, chances of that going ahead are interesting.

IOCs: https://community.snowflake.com/s/article/Communication-ID-0108977-Additional-Information

Snowflake admin users need to check their Snowflake environment, not sec departments check their on prem.

Snowflake Community

Join our community of data professionals to learn, connect, share and innovate together

❓ 😅
Five orgs have told me they are running incidents for Snowflake, where their data has been copied.

Snowflake: there is absolutely no cybersecurity incident.

Also Snowflake: Please run these commands and look for "threat activity" logins with the user agent "rapeflake" using this knowledge base article we haven't listed on our website.

https://community.snowflake.com/s/article/Communication-ID-0108977-Additional-Information

Snowflake Community

Join our community of data professionals to learn, connect, share and innovate together

Live Nation said its stolen database was hosted on Snowflake, a cloud storage and analytics company.

https://techcrunch.com/2024/05/31/live-nation-confirms-ticketmaster-was-hacked-says-personal-information-stolen-in-data-breach/

Live Nation confirms Ticketmaster was hacked, says personal information stolen in data breach | TechCrunch

Live Nation says its Ticketmaster subsidiary was hacked. A hacker claims to be selling 560 million customer records.

TechCrunch
I've now confirmed 6 major orgs running Snowflake cyber incidents, so I've made a theme song about Snowflake's response.

The deleted Hudson Rock post on Snowflake breach: https://web.archive.org/web/20240531140540/https://hudsonrock.com/blog/snowflake-massive-breach-access-through-infostealer-infection

For the record I don't think all the content is accurate - however Snowflake did have a security incident via their former employee, they have full IR stood up. They didn't follow their own best practices.

I also know multiple orgs who've had their full databases taken from Snowflake.

Snowflake, Cloud Storage Giant, Suffers Massive Breach: Hacker Confirms to Hudson Rock Access Through Infostealer Infection

Hudson Rock is able to confirm a massive breach at Snowflake was caused by credentials compromised via an Infostealer infection.

I wrote a blog on everything I know about the Snowflake situation https://doublepulsar.com/snowflake-at-central-of-worlds-largest-data-breach-939fc400912e
Snowflake at centre of world’s largest data breach - DoublePulsar

Cloud AI Data platform Snowflake are having a bad month. Due to teenager threat actors and cybersecurity of its own customers… and its own cybersecurity, too, in terms of optics. There are several…

DoublePulsar

The Snowflake authentication setup is terrible.

MFA can’t be enabled org wide, each user has to manually log in and enable it. There’s no policy to block users without MFA. And it uses Duo MFA rather than your orgs MFA. (You can bring your own MFA with SAML).

Also all users log in via a Snowflake domain, so you can just pull creds from info stealer marketplaces or logs.

That’s why they’re being targeted as a platform.

Hudson Rock have put out a statement saying a legal threat from Snowflake caused them to remove their blog. https://www.linkedin.com/posts/hudson-rock_activity-7203433945919578113-RH05 HT @mattburgess
Hudson Rock posted on LinkedIn

Hudson Rock posted images on LinkedIn

What Snowflake isn't saying about its customer data breaches | TechCrunch

As another Snowflake customer confirms a data breach, the cloud data company says its position "remains unchanged."

TechCrunch
One thing I didn't know until recently is Snowflake has a massive fanbase, Apple and Amiga style - if you critique Snowflake in any way people flip tables. The comments on my blog are fun. I mean, the clue is in the product name, really.

IMHO it's fair to call out Snowflake's authentication isn't very good - it's the worst SaaS MFA solution I've seen as it has no top level, easy switch for org wide MFA enforcement.

Combined with putting all customers under *.snowflakecomputing.com sub domain is why their customers are getting owned - infostealers are just full of creds ready to go.

I gather Snowflake are discussing changes to fix, don't tell the fanboys (and yes, they're all dudes).

Mandiant have informed 165 organisations they may have had data exfiltration from their Snowflake hosted databases

https://cloud.google.com/blog/topics/threat-intelligence/unc5537-snowflake-data-theft-extortion/

UNC5537 Targets Snowflake Customer Instances for Data Theft and Extortion | Google Cloud Blog

A campaign targeting Snowflake customer database instances with the intent of data theft and extortion.

Google Cloud Blog
Pure Storage Documentation portal

Kinda interesting - Mandiant notified Snowflake that over 100 customers had data exfil issues, and Snowflake’s share price immediately began to tank in sells offs - before the incident was made public.

✅ won a game of Call of Duty
✅ hacked the world’s largest companies
✅ used an infostealer

Can’t wait for these guys to have super secure Microsoft Recall, which is definitely encrypted from the user 🤪🤪🫡

https://www.bleepingcomputer.com/news/security/pure-storage-confirms-data-breach-after-snowflake-account-hack/

Pure Storage confirms data breach after Snowflake account hack

Pure Storage, a leading provider of cloud storage systems and services, confirmed on Monday that attackers breached its Snowflake workspace and gained access to what the company describes as telemetry information

BleepingComputer

Snowflake have told customers "We are also developing a plan to require our customers to implement advanced security controls, like multi-factor authentication (MFA) or network policies, especially for privileged Snowflake customer accounts."

Good! They also say the attack was "not caused by a vulnerability, misconfiguration, or breach of its product". Just happy little bad MFA.

Nice: "In a phone call this week, Jones (Snowflake CISO) told WIRED that Snowflake is working on giving its customers the ability to mandate that users of their accounts employ multifactor authentication going forward, “and then we’ll be looking in the future to [make the] default MFA,” he says."

This will be a great outcome for Snowflake customers and Snowflake itself. I know Snowflake got big mad at me for pointing it out, but that was a prime weakness in their MFA.

https://www.wired.com/story/epam-snowflake-ticketmaster-breach-shinyhunters/

Hackers Detail How They Allegedly Stole Ticketmaster Data From Snowflake

A ShinyHunters hacker tells WIRED that they gained access to Ticketmaster’s Snowflake cloud account—and others—by first breaching a third-party contractor.

WIRED
Snowflake, Cloud Storage Giant, Suffers Massive Breach: Hacker Confirms to Hudson Rock Access Through Infostealer Infection

Hudson Rock is able to confirm a massive breach at Snowflake was caused by credentials compromised via an Infostealer infection.

65 page PDF on searching for Snowflake malicious activity: https://services.google.com/fh/files/misc/snowflake-threat-hunting-guide.pdf
When Snowflake allows orgs to easily mandate MFA across their users, I plan to answer this forum post from 2019. https://community.snowflake.com/s/question/0D50Z00008ugjwISAQ/is-there-a-way-to-force-all-users-to-use-mfa
Snowflake Community

Join our community of data professionals to learn, connect, share and innovate together

Cisco has a look across the wider infostealer problem, using Snowflake as a jumping off point: https://blog.talosintelligence.com/infostealer-landscape-facilitates-breaches/
Snowflake isn’t an outlier, it’s the canary in the coal mine

By Nick Biasini with contributions from Kendall McKay and Guilherme Venere Headlines continue to roll in about the many implications and follow-on attacks originating from leaked and/or stolen credentials for the Snowflake cloud data platform. Adversaries obtained stolen login credentials for Snowflake accounts acquired via information-stealing malware and used

Cisco Talos Blog

I think SaaS providers who provide their own authentication have a responsibility to provide robust, *enforceable* MFA for their customers - so if an org wants all their users to require MFA, they can and it’s just an easy tick box.

Some SaaS providers aren’t doing this - - and it’s the reason infostealer logs are such a problem. Their angle is customer is solely responsible, but as a counterpoint: see how that is working out for Snowflake.

Snowflake have rolled out MFA changes:

- A new authentication policy that requires MFA for all users in a Snowflake account

- prompting for user-level MFA setup

- Snowflake Trust Center for monitoring adherence to MFA policies

This solves all the inherent product weaknesses from the prior setup, they did a good job.

https://www.snowflake.com/blog/snowflake-admins-enforce-mandatory-mfa/

@GossiTheDog It’s a good start! #AWS doesn’t even let admins force #IAM users to use #MFA !
@GossiTheDog all it took was a massive data breach that affected millions of people

@GossiTheDog if the SaaS vendors could provide SSO integration without having to pay extra, it will definitely help reduce these type of events. Stop putting paywalls on basic security features. Not all companies can afford to pay for enterprise licences to get SSO.

https://sso.tax/

The SSO Wall of Shame

A list of vendors that treat single sign-on as a luxury feature, not a core security requirement.

The SSO Wall of Shame
@GossiTheDog "I would also like to have a setting that enforces MFA. Hopefully this can be implemented soon," said Bas... *two years ago*.

@zackwhittaker @GossiTheDog Exactly.

Blaming the users on their password hygiene, when you don't offer org admins any way to enforce SSO or MFA, is a bit ... let's say, not a great look

@GossiTheDog out of the memory hole it returns, lol
@GossiTheDog Do we know why it got removed? Did Snowflake C&D them? And if so, why, is there anything manifestly untrue in it? (Looked to me from the archive that HR was careful to state that this was all 'claimed by the hacker', not proven facts).
@GossiTheDog into the memory hole it goes

@GossiTheDog seems like misconfiguration if it can be fixed by forcing customers into using a different configuration…

but I guess it wasn’t a misconfiguration of snowflake’s own production account(s) at least

@GossiTheDog Oh my "It's a happy little cyber incident!" is now stuck in my head.

@GossiTheDog people who still use #Windows11 should be legally barred from doing anything re: #ITsec.

  • except when they only use it as target for hacking and research and not their professional machine where they handle actual data on!
@GossiTheDog SEC be like 👀👀👀
@GossiTheDog some of their telemetry data has and it doesn’t contain customer info…

@GossiTheDog
@Johnhultquist
Even if Snowflake had "proper" MFA authentication across the board, would that have significantly changed the outcome?

That is, if an info stealer is what's at play, and it steals my authentication cookie, I can't really rest at night knowing that MFA is keeping me safe, right?

@wdormann @GossiTheDog @Johnhultquist I think this is why we need device bound credentials, but that spec is still in progress. MFA at least helps with people brute forcing passwords, but rate limiting would help for that too

@jawnsy @wdormann @GossiTheDog @Johnhultquist

How do you device bound credentials? I have only heard of IP bounding them (so if it’s suddenly being used from a different location a new login is required)
And that is already being used by some companies afaik.

@Euph0r14 @wdormann @GossiTheDog @Johnhultquist I don't think you can until this (or something like it) is added to the web standards https://github.com/WICG/dbsc
GitHub - WICG/dbsc

Contribute to WICG/dbsc development by creating an account on GitHub.

GitHub
@GossiTheDog 😱 that seems like a lot

@GossiTheDog

> putting all customers under *.snowflakecomputing.com sub domain

How else would you have them handle it? Nearly all SaaS providers do exactly this.

A popular one you might have come across is *.github.io.

(Not a fanboy; I just work for another SaaS vendor and would like to know if we’re doing anything terribly wrong.)

@GossiTheDog Yeah we were setting up MFA for a Snowflake SSO break-glass account and the enrollment process didn't work ("success" but then locked out), had 2 more people try and it didn't work for them either (same issue), this was months ago so it's probably fixed but I wouldn't be surprised if enrollment friction played a role
@GossiTheDog Couldn't possibly be related... /s
@GossiTheDog maybe it was a bad idea to just dump all the data into one big pile with just a big sign saying “bad guys stay out” defending it.
@GossiTheDog so has their summit actually been going on for the last 3 days? Not sure I'm willing to sign into the hellsite to look for the hashtag.
@GossiTheDog Similar to the old hardware reverse engineering practices where the people touching the original hardware and the engineers designing the clone are never the same people, I wonder if there are quiet companies out there who buy stuff like this and then sell a “customer lead db” to competitor stores in a way where the competitor can legally pretend it isn’t stolen data.
@GossiTheDog @mattburgess That's good considering Hudson Rock's post was completely wrong. Not to mention calling out a specific employee that they believed had been compromised is not only unprofessional, but completely inappropriate. If it had been true, can you imagine the psychological toll of knowing that you were inadvertently responsible for something like that?
@GossiTheDog I don't think that's completely accurate. I login to a couple of Snowflake accounts with my organisational SSO which includes our standard MFA.
@GossiTheDog Our (fairly large) org uses SAML federation, with MFA enforced.
@GossiTheDog I interviewed with them couple months ago, on cloud infra team. Interviewers seemed anxious when I started asking them on security posture practices and procedures. They don’t have any org wide authorization mechanisms and focus only on cost optimization+some automation.
@GossiTheDog re: credential theft, MFA and Recall - what's one of the things that'll be saved? Those handy QR codes for TOTP...

@fencepost @GossiTheDog

Oh fuu....you're right! This just keeps getting worse and worse.

I got to this point in your write-up, and I don't think it can be emphasized enough:

Note that in the age of SaaS, your providers will throw you under the bus to save themselves. When you transfer your security risk to a provider, they don’t accept your risk — they just take the money.[EDIT: typo]

@GossiTheDog

@GossiTheDog Might be a coincidence, but Prolific (paid study/research site) has just temporarily paused pay outs until further notice due to a cryptically worded “possible cyber security incident targeting users”.