For those who aren’t aware, Microsoft have decided to bake essentially an infostealer into base Windows OS and enable by default.
From the Microsoft FAQ: “Note that Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers."
Info is stored locally - but rather than something like Redline stealing your local browser password vault, now they can just steal the last 3 months of everything you’ve typed and viewed in one database.
I've written up my thoughts on the Copilot Recall feature in Microsoft Copilot+ PCs
I think it will enable fraud and endanger users, and is not the sign of a company who are committed to security first.
Copilot+ Recall has been enabled by default globally in Microsoft Intune managed users, for businesses.
You need to enable DisableAIDataAnalysis to switch it off. https://learn.microsoft.com/en-us/windows/client-management/manage-recall
Two quick updates -
A) if you disallow recording of a website in Control Panel or GPO, in Chrome it is still recorded - disallow recording only works in Edge browser
B) Firefox and Tor Browser is recorded always, including in private mode - the exception is Hollywood DRM’d videos
I got ahold of the Copilot+ software.
Recall uses a bunch of services themed CAP - Core AI Platform. Enabled by default.
It spits constant screenshots (the product brands then “snapshots”, but they’re hooked screenshots) into the current user’s AppData as part of image storage.
The NPU processes them and extracts text, into a database file.
The database is SQLite, and you can access it as the user including programmatically. It 100% does not need physical access and can be stolen.
And if you didn’t believe me.. found this on TikTok.
There’s an MSFT employee in the background saying “I don’t know if the team is going to be very happy…”
They should probably be transparent about it, rather than telling BBC News you’d need to be physically at the PC to hack it (not true). Just a thought.
So the code underpinning Copilot+ Recall includes a whole bunch of Azure AI backend code, which has ended up in the Windows OS. It also has a ton of API hooks for user activity monitoring.
Apps themselves can also search and make themselves more searchable.
It opens a lot of attack surface.
The semantic search element is fun.
They really went all in with this and it will have profound negative implications for the safety of people who use Microsoft Windows.
If you want to know where tech companies are with AI safety, know Microsoft Recall won’t record screenshots of DRM’d movies..
..but will record screenshots of your financial records and WhatsApp messages, as corporate interests were prioritised over user safety.
And it’s enabled by default.
Copilot+ Recall feature pop quiz:
You deal with a sensitive matter on my Windows PC. E.g. an email you delete. Does Copilot Recall still store the deleted email?
Answer: yes. There's no feature to delete screenshots of things you delete while using your PC. You would have to remember to go and purge screenshots that Recall makes every few seconds.
If you or a friend use disappearing messages in WhatsApp, Signal etc, it is recorded regardless.
It comes up a lot as people are rightly confused, but if you wonder what problem Microsoft are trying to solve with Recall:
It isn't them being evil, it's business leaders who are middle aged and can't remember what they're doing driving decision making about which problems to solve.
A huge amount of business leaders are dudes who have no idea what the fuck is happening. This leads to the Recall feature.
Microsoft exists in and is driven by that bubble.
Some screenshots of Recall's SQLite database here: https://mastodon.social/@detective/112513529733646088
Just to clarify, I can access it without SYSTEM too. Microsoft are about to set cybersecurity back a decade by empowering cyber criminals via poor AI safety. Feature ships in a few weeks.
The latest Risky Business episode on Recall is good, but one small correction - it doesn’t need SYSTEM rights.
Here’s a video of two MSFT employees gaining access to the Recall database folder - with SQLite database right there. Watch their hacking skills. (You don’t need to go this length as an attacker, either). Cc @riskybusiness
I’m not being hyperbolic when I say this is the dumbest cybersecurity move in a decade. Good luck to my parents safely using their PC.
Stealing everything you’ve ever typed or viewed on your own Windows PC is now possible with two lines of code — inside the Copilot+ Recall disaster.
My look at the feature, FAQs from the community etc
I wrote a piece recently about Copilot+ Recall, a new Microsoft Windows 11 feature which — in the words of Microsoft CEO Satya Nadella- takes “screenshots” of your PC constantly, and makes it into an…
I think they have AI so they don't have to think
This tutorial will show you how to completely disable the Windows Copilot preview feature and remove Copilot from the taskbar, Windows Search, and Microsoft Edge for all users in Windows 11 and Windows 10. Copilot in Windows provides centralized generative AI assistance to your users right from...
@GossiTheDog isn't it like the other "secure" directories? They are quite secure unless you visit them once and click continue on the shield. And once that is done, the protection is gone for good.
But hey, they warned you when you clicked continue. It's not like it's reasonable to expect it to only apply to the current instance of explorer or anything.
This ist also worth noticing:
According to Axios Microsoft is "exploring if there are ways that make sense to allow the feature to work across devices."
https://www.axios.com/2024/05/21/microsoft-windows-11-ai-recall-copilot-pc
So they republished a press release or press statement?
Last time I checked that was called Public Relations not Journalism 🫤
@janneoksanen @GossiTheDog exactly. Windows file search is great for small folders, for big ones it returns either nothing or everything. And Mac internal search has been getting worse.
It would be a great use of this new computing power.
And if someone wants to opt in to have their email, texts, etc also searchable, fine. I too have wasted time trying to find a message, unsure what platform it was in. But when i delete something I want it to be gone.
Interested layperson here:
From what you've seen so far, can you draw a conclusion that a computer with recall running sends more data than usual to Microsoft, maybe "disguised" as part of diagnostics data for example.
I wonder if recall is a fishing expedition in task mining and if so, how Microsoft, despite having promised that recall does not run in the cloud, could get the data nevertheless.
I started my work life in accounting and computers have made so many aspects of that work so much easier.
But I am at the point where I think it’s time to burn all this shit to the ground and get out the paper ledgers and 10 keys again.
I swear, every other day it’s another shit-tastic addition to some heavily used piece of software that no one asked for, is poorly made, and creates massive security holes.
@cafechatnoir I'm in my mid-50s, and believe me, that reflex is getting oooold. I'm sure I remember consumer affairs programmes running stories about computerised billing system SNAFUs in the 1970s.
"The lights had probably gone."
"So had the stairs!"
Small print doesn't really cut it, though, it needs to be informed consent. The victim, sorry 'data subject' needs to *genuinely* know & understand what they're agreeing to, and IIRC the directive rules out hiding significant detail in "paragraph 3, subsection 42" of agreements no-one ever reads.
@BibbleCo @8tpercent @GossiTheDog
But lets be honest - all of us living under the GDPR protections know how "informed consent" is handled by big companies.
[ ] by using this feature I accept the additions to the <privacy policy>
With the policy linked and no-one ever going to actually read all 300 pages of it or if they do understand the content.
Kevin Beaumont
AI6YR Ben
Why Not Zoidberg? 🦑
The Witch of Crow Briar
James 🦉 #FBPE 
v̾i̾t̾r̾i̾o̾l̾i̾x̾
2xfo
Patrick Gray
Jonly
Luuk
Paul Shread
Cassandrich
Michael Weiss
Random Damage 🌻
gigantos
docht
Parade du Grotesque 💀
Simon Zerafa
Salaryman
Ricky Boone
Jack Linke 🦄
Alun Jones
Maddie (MIGRATING SOON)
Emma (IPG)
Petrus Hilarius
Ric Harvey 🇪🇺🌍💚
Chris Merkel 🐀👨🏼🍳
sed
pgcd
Jared White (ResistanceNet ✊)
Janne Oksanen
Steven Bodzin
shane - powered by spite
Chris
Play Ball and Fight Fascists
BibbleCo
sadmac356
Jason Haar 
Fellows
Clayfoot
8tpercent
Heals 
P Stewart
Captain of the SS El Faro
Noah Cook