For those who aren’t aware, Microsoft have decided to bake essentially an infostealer into base Windows OS and enable by default.
From the Microsoft FAQ: “Note that Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers."
Info is stored locally - but rather than something like Redline stealing your local browser password vault, now they can just steal the last 3 months of everything you’ve typed and viewed in one database.
I've written up my thoughts on the Copilot Recall feature in Microsoft Copilot+ PCs
I think it will enable fraud and endanger users, and is not the sign of a company who are committed to security first.
Copilot+ Recall has been enabled by default globally in Microsoft Intune managed users, for businesses.
You need to enable DisableAIDataAnalysis to switch it off. https://learn.microsoft.com/en-us/windows/client-management/manage-recall
Two quick updates -
A) if you disallow recording of a website in Control Panel or GPO, in Chrome it is still recorded - disallow recording only works in Edge browser
B) Firefox and Tor Browser is recorded always, including in private mode - the exception is Hollywood DRM’d videos
I got ahold of the Copilot+ software.
Recall uses a bunch of services themed CAP - Core AI Platform. Enabled by default.
It spits constant screenshots (the product brands then “snapshots”, but they’re hooked screenshots) into the current user’s AppData as part of image storage.
The NPU processes them and extracts text, into a database file.
The database is SQLite, and you can access it as the user including programmatically. It 100% does not need physical access and can be stolen.
And if you didn’t believe me.. found this on TikTok.
There’s an MSFT employee in the background saying “I don’t know if the team is going to be very happy…”
They should probably be transparent about it, rather than telling BBC News you’d need to be physically at the PC to hack it (not true). Just a thought.
So the code underpinning Copilot+ Recall includes a whole bunch of Azure AI backend code, which has ended up in the Windows OS. It also has a ton of API hooks for user activity monitoring.
Apps themselves can also search and make themselves more searchable.
It opens a lot of attack surface.
The semantic search element is fun.
They really went all in with this and it will have profound negative implications for the safety of people who use Microsoft Windows.
If you want to know where tech companies are with AI safety, know Microsoft Recall won’t record screenshots of DRM’d movies..
..but will record screenshots of your financial records and WhatsApp messages, as corporate interests were prioritised over user safety.
And it’s enabled by default.
I started my work life in accounting and computers have made so many aspects of that work so much easier.
But I am at the point where I think it’s time to burn all this shit to the ground and get out the paper ledgers and 10 keys again.
I swear, every other day it’s another shit-tastic addition to some heavily used piece of software that no one asked for, is poorly made, and creates massive security holes.
@cafechatnoir I'm in my mid-50s, and believe me, that reflex is getting oooold. I'm sure I remember consumer affairs programmes running stories about computerised billing system SNAFUs in the 1970s.
"The lights had probably gone."
"So had the stairs!"
Small print doesn't really cut it, though, it needs to be informed consent. The victim, sorry 'data subject' needs to *genuinely* know & understand what they're agreeing to, and IIRC the directive rules out hiding significant detail in "paragraph 3, subsection 42" of agreements no-one ever reads.
@BibbleCo @8tpercent @GossiTheDog
But lets be honest - all of us living under the GDPR protections know how "informed consent" is handled by big companies.
[ ] by using this feature I accept the additions to the <privacy policy>
With the policy linked and no-one ever going to actually read all 300 pages of it or if they do understand the content.
@seattleswiftie @GossiTheDog "Shocked. Shocked I am, that gambling should be taking place on these premises!"
"Your winnings, Captain."
"Oh, thank you."
Consider...
This is a "feature" M$ held worthy of letting us know about cause they expected we'd want it...
Imagine what's in Windoze they haven't let us know about to avoid expected reaction...
🤔🤔🤔
Scott Hanselman
@GossiTheDog the dude in that TikTok is Scott Hanselman. He's active on Mastodon on hachyderm.io
He's generally friendly.
@GossiTheDog “Hope that helps”
More than you know mate. More than you know.
@GossiTheDog I never doubted for a second that it was actually being done on the NPU.
What I'll _never_ believe is that they'll not be using that stuff to monetize their users. More banners spammed to users, more value for shareholders.
It worths remembering Microsoft's main strategy: Embrace, Extend, Extinguish
This time the target is humans, because fewer humans to pay, more value for the shareholder.
Kevin Beaumont
Chris
Play Ball and Fight Fascists
BibbleCo
sadmac356
Jason Haar 
Fellows
Clayfoot
8tpercent
Heals 
P Stewart
Captain of the SS El Faro
Lukas Brausch
Alan Miller 
🇺🇦
SeattleSwiftie
kauer
Michael Weiss
Magnesium
Interiorce
Matt
lupus_blackfur
snowyfox
Graham Sutherland / Polynomial
Luka Rubinjoni
Ignas Kiela
Quadruple
Kyle Green
BobHy
DarkCyberman
DHeadshot's Alt
Nischay
Demiurg
waffles 
Marcus Adams
Gavin Jones 
Thibault D.
Matti Järvinen
Scott Hanselman 👸🏽🐝🌮
Tane Piper ⁂
Andrew Prillaman
Gaëtan Perrault
Ian Robinson
DelegateVoid
unlucio 🌍 