Copilot+ Recall has been enabled by default globally in Microsoft Intune managed users, for businesses.

You need to enable DisableAIDataAnalysis to switch it off. https://learn.microsoft.com/en-us/windows/client-management/manage-recall

Manage Recall for Windows clients

Learn how to manage Recall for commercial environments and about Recall features.

Here’s Copilot+ Recall search in action, showing instant text based search finding a WhatsApp chat and a PDF from 6 months ago being viewed on screen.

Two quick updates -

A) if you disallow recording of a website in Control Panel or GPO, in Chrome it is still recorded - disallow recording only works in Edge browser

B) Firefox and Tor Browser is recorded always, including in private mode - the exception is Hollywood DRM’d videos

I got ahold of the Copilot+ software.

Recall uses a bunch of services themed CAP - Core AI Platform. Enabled by default.

It spits constant screenshots (the product brands then “snapshots”, but they’re hooked screenshots) into the current user’s AppData as part of image storage.

The NPU processes them and extracts text, into a database file.

The database is SQLite, and you can access it as the user including programmatically. It 100% does not need physical access and can be stolen.

And if you didn’t believe me.. found this on TikTok.

There’s an MSFT employee in the background saying “I don’t know if the team is going to be very happy…”

They should probably be transparent about it, rather than telling BBC News you’d need to be physically at the PC to hack it (not true). Just a thought.

I ponder if Microsoft's engineers are following the SQLite code of ethics, since they're using it in Windows OS with Copilot+ Recall? :D https://sqlite.org/codeofethics.html
Code Of Ethics

So the code underpinning Copilot+ Recall includes a whole bunch of Azure AI backend code, which has ended up in the Windows OS. It also has a ton of API hooks for user activity monitoring.

Apps themselves can also search and make themselves more searchable.

It opens a lot of attack surface.

The semantic search element is fun.

They really went all in with this and it will have profound negative implications for the safety of people who use Microsoft Windows.

If you want to know where tech companies are with AI safety, know Microsoft Recall won’t record screenshots of DRM’d movies..

..but will record screenshots of your financial records and WhatsApp messages, as corporate interests were prioritised over user safety.

And it’s enabled by default.

I’ve managed to get Recall working in full on a non-Copilot+ system, without an NPU. Will accelerate testing.

Copilot+ Recall feature pop quiz:

You deal with a sensitive matter on my Windows PC. E.g. an email you delete. Does Copilot Recall still store the deleted email?

Answer: yes. There's no feature to delete screenshots of things you delete while using your PC. You would have to remember to go and purge screenshots that Recall makes every few seconds.

If you or a friend use disappearing messages in WhatsApp, Signal etc, it is recorded regardless.

It comes up a lot as people are rightly confused, but if you wonder what problem Microsoft are trying to solve with Recall:

It isn't them being evil, it's business leaders who are middle aged and can't remember what they're doing driving decision making about which problems to solve.

A huge amount of business leaders are dudes who have no idea what the fuck is happening. This leads to the Recall feature.

Microsoft exists in and is driven by that bubble.

I asked Microsoft Copilot to write a song about Copilot+ Recall.
Managed to find out how BBC News printed in a headline story that it was not possible to steal Recall data without being physically at the device (which is false) - this is from the journalist:

Some screenshots of Recall's SQLite database here: https://mastodon.social/@detective/112513529733646088

Just to clarify, I can access it without SYSTEM too. Microsoft are about to set cybersecurity back a decade by empowering cyber criminals via poor AI safety. Feature ships in a few weeks.

The latest Risky Business episode on Recall is good, but one small correction - it doesn’t need SYSTEM rights.

Here’s a video of two MSFT employees gaining access to the Recall database folder - with SQLite database right there. Watch their hacking skills. (You don’t need to go this length as an attacker, either). Cc @riskybusiness

I’m not being hyperbolic when I say this is the dumbest cybersecurity move in a decade. Good luck to my parents safely using their PC.

Stealing everything you’ve ever typed or viewed on your own Windows PC is now possible with two lines of code — inside the Copilot+ Recall disaster.

My look at the feature, FAQs from the community etc

https://doublepulsar.com/recall-stealing-everything-youve-ever-typed-or-viewed-on-your-own-windows-pc-is-now-possible-da3e12e9465e

Stealing everything you’ve ever typed or viewed on your own Windows PC is now possible with two lines of code — inside the Copilot+ Recall disaster.

I wrote a piece recently about Copilot+ Recall, a new Microsoft Windows 11 feature which — in the words of Microsoft CEO Satya Nadella- takes “screenshots” of your PC constantly, and makes it into an…

DoublePulsar

this is the out of box experience for Windows 11's new Recall feature on Copilot+ PCs. It's enabled by default during setup and you can't disable it directly here. There is an option to tick "open Settings after setup completes so I can manage my Recall preferences" instead.

HT @tomwarren

You allow BYOD so people can pick up webmail and such. It’s okay, because when they leave you revoke their access, and your MDM removes all business data from the machine ✅

What the employee does: opens Recall, searches their email, files etc and pastes the data elsewhere.

Nothing is removed from Recall, as it is a photographic memory of everything the former employee did.

Just in time for Copilot+ Recall!

Security and privacy researchers - You can now install Copilot+ Recall on any ARM hardware (doesn’t need an NPU) or in Azure VMs.

Guide from @detective

The devices launch THIS MONTH to customers so I suggest people look at this.

https://github.com/thebookisclosed/AmperageKit

GitHub - thebookisclosed/AmperageKit: One stop shop for enabling Recall in Windows 11 version 24H2 on unsupported devices

One stop shop for enabling Recall in Windows 11 version 24H2 on unsupported devices - thebookisclosed/AmperageKit

GitHub
Nvidia just announced that Copilot+ and Recall are coming to AMD systems. https://www.theverge.com/2024/6/2/24169568/microsoft-copilot-plus-gaming-pc-nvidia-amd
Nvidia and AMD are bringing Microsoft’s Copilot Plus AI features to gaming laptops

Asus and MSI are launching AMD- and Nvidia-powered gaming laptops that include Microsoft’s Copilot Plus AI features.

The Verge
Somebody made a tool called Total Recall to dump Recall database and screenshots. https://x.com/xaitax/status/1797349055917416457?s=46
Alex (@xaitax) on X

Will release TotalRecall in a few days. Loads to play with and to work on. Thank you @GossiTheDog for the inspiration! #WindowsRecall #CyberSecurity #Microsoft #TotalRecall

X (formerly Twitter)

Recent DHS published report handed to the US President which said it had "identified a series of Microsoft operational and strategic decisions that collectively pointed to a corporate culture that deprioritized enterprise security investments and rigorous risk management"

Microsoft: let’s use AI to screenshot everything users do every 5 seconds, OCR the screenshots, make it searchable and store it in AppData!

Searching Recall database for passwords with @awakecoding
🫡

If anybody is wondering if you can enable Recall on a machine remotely without Copilot+ hardware support - yep.

I’ve also found a way to disable the tray icon.

I went and looked at YouTube for Recall to get out of the echo chamber and I can only find one positive video. Even the people at the event are slating it, including people with media provided Copilot+ PCs.

There’s some content creators who’ve realised it records their credit cards, so they’re making videos of their cards going walkies.

It’s going to be interesting to see how Microsoft get out of this one. They may have contractual commitments to ship Recall with external parties.

I thought they were risking crashing the Copilot brand with this one, but I was wrong looking at the videos and comments on them - I think they’re crashing the Windows consumer brand.

The reaction to photographic memory of what people do at home has - you’ll be surprised to know - not been seen as a reason to buy a device, but a reason why not to.

Windows Central, about the only outlet giving Recall positive coverage and having articles tweeted by Microsoft staff - have updated their take after being hands on with a device. https://www.windowscentral.com/software-apps/windows-11/microsoft-should-recall-windows-recall-security-researcher-finds-microsofts-new-ai-tool-woefully-insecure
"Microsoft should recall Windows Recall" — Security researcher discovers Microsoft's new AI tool is woefully insecure

The security story around Windows Recall hits a brick wall as it's discovered the data it collects is unencrypted.

Windows Central

Microsoft has been declining to comment on criticism of Recall for a week - but they have apparently told a journalist off the record at Future that changes will be made before Copilot+ devices drop in the coming days.

This may include an attempt to invalidate researcher criticism, we’ll see.

WIRED has a piece about Total Recall, a now released tool which dumps keypresses, text and screenshots (they’re JPEGs) from Microsoft Recall

https://www.wired.com/story/total-recall-windows-recall-ai/

Total Recall software by @xaitax https://github.com/xaitax/TotalRecall

Example search for ‘password’:

🪟 Captured Windows: 133
📸 Images Taken: 36
🔍 Search results for 'password': 22

📄 Summary of the extraction is available in the file:
C:\Users\alex\Downloads\TotalRecall\2024-06-04-13-49_Recall_Extraction\TotalRecall.txt

This Hacker Tool Extracts All the Data Collected by Windows’ New Recall AI

Windows Recall takes a screenshot every five seconds. Cybersecurity researchers say the system is simple to abuse—and one ethical hacker has already built a tool to show how easy it really is.

WIRED

I hadn’t been aware until today of the external reaction to Recall. Holy shit. Tim Apple must be pleased.

Everything from media coverage to YouTube to TikTok is largely negative. All the comments are negative.

These videos have tens of millions of views and hundreds of thousands of comments.

I knew it would be bad but.. it’s worse. I’ve spent hours looking at the sentiment and.. well, they probably would have got better coverage from launching an NFT of pregnant Clippy.

A key element of Recall is Microsoft say only you can access your Recall, it is per user.

ArsTechnica enabled Recall on Windows 11 box and tested the claim. By logging in as another user they could access the database and screenshots.

https://arstechnica.com/ai/2024/06/windows-recall-demands-an-extraordinary-level-of-trust-that-microsoft-hasnt-earned/

Windows Recall demands an extraordinary level of trust that Microsoft hasn’t earned

Op-ed: The risks to Recall are way too high for security to be secondary.

Ars Technica

If you want to know how Microsoft have got themselves into this giant mess with Recall, here’s what the documentation says between the lines:

you, the customer, are a simpleton who doesn’t want to be an AI genius yet. Have a caveman mode.

Recall and Copilot+ is also coming to ASUS systems, including AMD, in a deal with Microsoft.

ASUS Announces Complete Portfolio of AI-Powered Copilot+ PCs https://www.asus.com/us/news/pnm9tg6qccql6ern/

Nvidia announced they are bringing Copilot+ and Recall to PCs, in a deal with Microsoft: https://www.theverge.com/2024/6/2/24169568/microsoft-copilot-plus-gaming-pc-nvidia-amd

ASUS Announces Complete Portfolio of AI-Powered Copilot+ PCs at Computex 2024

Fremont, Calif., June 3, 2024 - ASUS today ushered in a new era of Copilot+ PCs — featuring advanced AI capability with 45+ TOPS NPU AI engines — during its Always Incredible

ASUS

Three Copilot+ Recall questions that keep coming up.

Q. Can you alter the Recall history?

A. Yes. You can change the OCR database and change the screenshots as the logged in user or as software running as the local user. There is no audit log of changes.

Q. Are they snapshots, as Microsoft says, or screenshots?

A. They are just screenshots, jpegs.

Q. What is to stop apps on your machine accessing your Recall covertly?
A. Nothing. There is no audit log of access.

.@awakecoding becomes the latest person reverse engineering Microsoft Recall https://x.com/awakecoding/status/1798168395583746216
Marc-André Moreau (@awakecoding) on X

@MalwareJake Recall is a melting pot of everything wrong with modern Windows: Per-user app and settings MSIX app setting virtualization Intune MDM per-user policies WinRT generated proxy code Enabled by default, opt-out If you hate it, it's in there, I tell you

X (formerly Twitter)

If anybody is wondering what Microsoft's reaction to any of the Copilot+ Recall concerns are, they're continuing to decline comment to every media outlet.

I've seen comments MS staff have been given for enterprise customers, which are nonsense handwaving.

Product ships live on devices from Dell, Lenovo etc this month. https://x.com/zacbowden/status/1798221879741931847

Zac Bowden (@zacbowden) on X

Microsoft has gone radio silent on Windows Recall.

X (formerly Twitter)
As @tiraniddo rightly points out, anybody can programmatically reach the Recall database without admin rights. https://infosec.exchange/@tiraniddo/112566044174482506
James Forshaw :donor: (@tiraniddo@infosec.exchange)

Damn, I really thought the Recall database security would at least be, you know, secure. Turns out Microsoft did pretty much what I blogged about for WindowsApps, except you need to find a specific WIN://SYSAPPID instead. So to bypass the security just get the token for the AIXHost.exe process, then impersonate that and you can access the database, no admin required. Or, as the files are owned by the user, just grant yourself access using icacls etc :D

Infosec Exchange
TotalRecall has been updated to exfiltrate Recall database and screenshots without needing admin rights: https://github.com/xaitax/TotalRecall
GitHub - xaitax/TotalRecall: This tool extracts and displays data from the Recall feature in Windows 11, providing an easy way to access information about your PC's activity snapshots.

This tool extracts and displays data from the Recall feature in Windows 11, providing an easy way to access information about your PC's activity snapshots. - xaitax/TotalRecall

GitHub

You can now remotely dump Recall data and screenshots over the internet from Linux etc. Changes in flight for parsing data too.

https://github.com/Pennyw0rth/NetExec/pull/335

Add Recall module for dumping all users Microsoft Recall DBs & screenshots by Marshall-Hallenbeck · Pull Request #335 · Pennyw0rth/NetExec

Gets all users Recall folders and dumps them, then renames screenshots to include .jpg (unnecessary but helpful). I cherry-picked the download_folder functionality from #320 and then improved it du...

GitHub
YouTubers are continuing to have fun with Recall

Turns out speaking out works.

Microsoft are making significant changes to Recall, including making it specifically opt in, requiring Windows Hello face scanning to activate and use it, and actually encrypting the database.

There are obviously going to be devils in the details - potentially big ones.

Microsoft needs to commit to not trying to sneak users to enable it in the future, and it needs turning off by default in Group Policy and Intune for enterprise orgs.

https://www.theverge.com/2024/6/7/24173499/microsoft-windows-recall-response-security-concerns

Windows won’t take screenshots of everything you do after all — unless you opt in

Microsoft is making its controversial AI-powered Recall feature optional. The changes come after security experts warned the feature could be a disaster for cybersecurity.

The Verge

Obviously, I recommend you do not enable Recall, and you tell your family not to enable it too.

It’s still labelled Preview, and I’ll believe it is encrypted when I see it.

There are obviously serious governance and security failures at Microsoft around how this played out that need to be investigated, and suggests they are not serious about AI safety.

Microsoft President Brad Smith is going to be grilled by US gov next week. https://therecord.media/microsoft-reverses-course-recall-opt-in
Microsoft reverses course, makes Recall feature opt-in only after security backlash

Recall allows the company’s new line of Windows 11 Copilot+ devices to screenshot every action a person takes on their PC.

I should be transparent btw that I took Satya and Charlie’s commitment to security at face value too - I even published a blog on it backing that up - and I have concerns (it isn’t just me).

They’re now going to have to win trust back about winning trust back.

I know somebody at a retailer in Europe that is selling Copilot+ PCs. They’ve had fewer than a thousand preorders through to customers.

In relative terms, for them it’s about as successful as Suicide Squad Kill The Justice League.

A reminder that a few weeks ago at RSA, Microsoft signed CISA's Secure By Design pledge... and then shipped an enabled by design keylogger that OCRs your screen constantly into AppData.

Edit: I should say that's less a reflection on Microsoft and more a reflection on CISA's Secure By Design pledge.. it's a good idea, but the scope is extremely limited.

@GossiTheDog

You didn't expect them to actually follow through with the pledge though, did you?

That's just for PR! Line must go up! -Microsoft CEO

@GossiTheDog

In all fairness, a vulnerability is usually unintended behavior in a specific piece of software. This was intentional, so there's no vulnerability here.

I never thought I'd see the day where "It's not a bug, it's a feature" was translated into the infosec realm, but Microsoft found a way.

@GossiTheDog tell me you don't get secure by design..
@GossiTheDog I can't keep track of which of the batman-extended-universe games flopped at this point so I'm going to assume the sales numbers are bad.
@GossiTheDog MS Bob sold 58k units, copilot+ has a while to go. looking objectively to be the worst product launch in microsoft history
@miunau @GossiTheDog and Bob went on to become the basis for Microsoft Agent, which led to Clippy, Merlin, and Rover (the XP search dog). a solid success story compared to this.
@gsuberland @GossiTheDog i mean it got us pregnant clippy so it's clearly a win
@miunau @gsuberland @GossiTheDog No, rule 34 got us that
@drwho @miunau @GossiTheDog *puts philosophy hat on* ah, but to make porn of what exists, the subject must first exist

@GossiTheDog which is super annoying to me.

They had the chance to finally sell a powerful #Windows laptop to compete with Apple on battery life and performance.

And what did they do? They made it all about the NPU, so now the only reason to buy it is a feature nobody wants or understands.

@GossiTheDog about the same here in norway from what I gathered.
Less than 1000 sold so far (no idea if this was across the chain, region or that store specifically)

@GossiTheDog
I really want to know how they're going to square this new and improved concept with the fact that it's going to literally be used in abusive situations to get a better hold on the victims.

I realize this isn't a solvable problem, but the fact that it never even apparently showed up on their radar as a potential abuse is...upsetting to say the least.

@GossiTheDog Makes sense as Copilot is still not available in the European Economic Area, which is fine by me.
@GossiTheDog Wyden doing the lord's work, roasting big tech over the fucking coals.
@GossiTheDog I read that as “Bard Smith” and imagined the Google AI attending Congressional hearings and what a poor job it would do 😆
@GossiTheDog notes on social media....but WHICH social media? Was it chad xitter or that rinky dink mastodon /s

@GossiTheDog

@csoghoian should see this thread, but their profile says they're on "social media hiatus"

@GossiTheDog it’s a good article and you did good using your social media megaphone to call attention to this
@GossiTheDog Would you ever testify if asked?

@GossiTheDog heavily reminded of https://en.wikipedia.org/wiki/Office_of_Strategic_Influence

Once they admit they're doing it, it's quite unlikely to ever actually disappear.

Even if it's disabled, Windows Update will *totally accidentally* enable it at some point. Same for encryption.

Office of Strategic Influence - Wikipedia

@GossiTheDog To still be useful encryption implies there will be a decryption key. One for the user and possibly the comeback for _NSAKEY 😀

@GossiTheDog Recall's gonna happen. Either accept that Big Brother has won, or use something else.

If you're one of the hapless slobs who can't use something else because your boss loves Big Brother, my condolences.

@GossiTheDog They’re not serious about *any* kind of safety or security TBH

@GossiTheDog you nailed it with "serious governance failures" and quoting by the way tha latest CISA report.

https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf

@GossiTheDog I wouldn't trust them to keep their promise.

@lulu @GossiTheDog There's so many examples of corps making solemn promises to not try that again only to quietly change back to whatever egregious bullshit they wanted 6 weeks, or 6 months, or even a few years later.

These are just not good people. They have the ethics of very hungry and spoiled retrievers. They exploit and find workarounds for a living!

@cjpaloma @lulu @GossiTheDog
It's less people than culture. If you're rewarded by changing things, then whoever is making the promise will eventually move to another role or leave, and another set of folks get to re-decide whether they agree with what's happened with Recall or not. "Never attribute to malice that which is adequately explained by stupidity"

@kladni @lulu @GossiTheDog

that explanation might look good to you on paper, but

Cultures are made from people. People doing certain things and not other things…people with ethics made from rotted fish heads, or folks with more developed senses of right and wrong.

Certain types of people are wanted at certain corporations and certain types are -willing to- make these kinds of usually pretty unethical decisions. It's both.

@cjpaloma @lulu @GossiTheDog
Your comment is reasonable. Though none of us can know for sure what lies in the hearts of those who are responsible here, and if ethical compromises were intentional or a byproduct of said culture. Of course it's people making up this culture, no argument there. In any case, the Recall feature is a disaster, and it's good news that the pushback has caused something to happen. The risk remains: things can get un-decided due to either bad ethics or poor incentives.
@GossiTheDog hooray! They're doing what they said they'd do in the first place!
@starchy @GossiTheDog
*They **said** they're going to do what they said they'd do in the first place!

@GossiTheDog Sounds like they finally ran this plan past legal, and after having to administer enough sedatives to drop a herd of elephants, the attorneys managed to stop screaming long enough to put put a basic list of Things That Must Happen.

FWIW, policy analysts cost about 1/10th what an attorney charges per-hour. That's actually a pretty standard FAFO tax, if you think about it.

@GossiTheDog Using the modern encrypted and attested VMs would be a massive step. I feel like that would require too much engineering effort for them to implement though.
@GossiTheDog Do these security changes impact its functionality in any way? Is there a reason they didn’t set it up this way right from the start? The original rollout seems like such a terrible self-inflicted wound — I don’t get how MS could have been so clueless.
@GossiTheDog Speaking out worked but seriously we should not have needed all that noise to begin with.
@GossiTheDog This might be the only page in Windows setup where there’s no primary blue button for the user to mindlessly click. I don’t know if MS can say “off by default” when there’s no default.
@GossiTheDog Better than nothing but why not make it even more explicit, like "remember this" and "remember next 5 minutes" button at taskbar, require explicit action from person to use that? Or even broadly, why that Recall feature has been pushed into people mouth at first place? Anyway, I will move to Linux soon.

@GossiTheDog
I love how people like this just *crumble* with fluffy words when it comes to security.

"This sounds like spyware, how can I make sure nobody accesses it?"
"The neat thing about that is, it's your data"
"Yes I know it's my data. But how can I make sure nobody accesses it?"
[This section was cut per request]

@GossiTheDog Either someone lied to him, or he’s lying to us.
@GossiTheDog The arrogance of this man is astonishing

@GossiTheDog

"All your secret are belong to us!"