For those who aren’t aware, Microsoft have decided to bake essentially an infostealer into base Windows OS and enable by default.
From the Microsoft FAQ: “Note that Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers."
Info is stored locally - but rather than something like Redline stealing your local browser password vault, now they can just steal the last 3 months of everything you’ve typed and viewed in one database.
I've written up my thoughts on the Copilot Recall feature in Microsoft Copilot+ PCs
I think it will enable fraud and endanger users, and is not the sign of a company who are committed to security first.
Copilot+ Recall has been enabled by default globally in Microsoft Intune managed users, for businesses.
You need to enable DisableAIDataAnalysis to switch it off. https://learn.microsoft.com/en-us/windows/client-management/manage-recall
Two quick updates -
A) if you disallow recording of a website in Control Panel or GPO, in Chrome it is still recorded - disallow recording only works in Edge browser
B) Firefox and Tor Browser is recorded always, including in private mode - the exception is Hollywood DRM’d videos
I got ahold of the Copilot+ software.
Recall uses a bunch of services themed CAP - Core AI Platform. Enabled by default.
It spits constant screenshots (the product brands then “snapshots”, but they’re hooked screenshots) into the current user’s AppData as part of image storage.
The NPU processes them and extracts text, into a database file.
The database is SQLite, and you can access it as the user including programmatically. It 100% does not need physical access and can be stolen.
And if you didn’t believe me.. found this on TikTok.
There’s an MSFT employee in the background saying “I don’t know if the team is going to be very happy…”
They should probably be transparent about it, rather than telling BBC News you’d need to be physically at the PC to hack it (not true). Just a thought.
Scott Hanselman
@GossiTheDog the dude in that TikTok is Scott Hanselman. He's active on Mastodon on hachyderm.io
He's generally friendly.
You have to admit that's quite an endorsement for SQLite. But maybe the SQLite license should be updated to deny its use in implementing a Torment Nexus.
@jonhendry @starchy @GossiTheDog
Given the politics of the SQLite project, I shudder to think what sort of thing they would allow/disallow.
@passenger @starchy @GossiTheDog
It's mostly just the one guy, isn't it?
@jonhendry @starchy @GossiTheDog
Richard Hipp, yeah. As with many projects, a lot of the grunt work of development was done by other people though.
My original comment was related to the notorious code of ethics which he got those other devs to pledge to while working on the project.
If you haven't read it, it's here:
https://sqlite.org/codeofethics.html
(Richard, if you're reading this toot, I deeply respect you as a database engineer, but also wtf?)
@Andres4NY @passenger @starchy @GossiTheDog
I mean, it's the rules for an order of monks, so in that context it makes sense there'd be a lot.
As a code of ethics for a software project... ehhhhh.
@Andres4NY @jonhendry @starchy @GossiTheDog
And none of those rules are "don't sexually harass people", despite that being the proximal reason why we're now doing codes of conduct. "Don't be a transphobe", "don't be a misogynist" and "don't be a racist" are also things I'd have thought to include.
But then, I'm not a literal saint, so what do I know?
@passenger @Andres4NY @starchy @GossiTheDog
Those probably could fit under various rules in a rather fuzzy and non-specific way.
I mean, “Do no wrong to anyone, and bear patiently wrongs done to yourself.” if diligently followed would probably cover all the things you mentioned.
Of course the problem is that the perpetrator probably doesn't think what they did WAS wrong, thus the need for specifics.
@mikebabcock @passenger It was meant as an ethical pledge, but the language is so overloaded that it reads as an introduction to a cult.
Having worked on the #ContributorConvenant v3 @ethicalsource and knowing a few profissional codes of ethic, maybe it is time I work on some like time.
@noplasticshower @GossiTheDog well https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf
The fuck-ups are bad and the incident response even worse so. Any org worth its salt would have blacklisted Microsoft and/or o365 as a vendor by now.
But that's okay, because they have a PR dept https://www.microsoft.com/en-us/security/blog/2024/05/03/security-above-all-else-expanding-microsofts-secure-future-initiative/ and people are actually eating it up.
@GossiTheDog they’re not even doing anything interesting like vectorising the scraped text for use in an on-board RAG type system? It’s just a DL OCR and plain old search?
Still can’t see why any business that is vaguely sane would allow this on their fleet, or why any personal user would either. It screams privacy accident waiting to happen.
@phel @GossiTheDog yeah, that’s been a persistent feeling for me too: ideally you should be able to get this information over accessibility APIs. And that would let you have much more granularity *and* access control.
Using OCR is a crutch admission to how badly data access in modern OS has gotten broken - that‘s what it really feels like to me.
Basically, this is a feature that:
- wastes processing power
- actively makes your personal info vunrable to theft (think of all the not-techy people who are gonna get scammed with this)
- benefeits random companies and scammers while actively hindering the user
- gives personal info to Copilot, and if it trains off user inputs like ChatGPT does, then using it literally hands your personal info, passwords, etc. to a database where anyone who knows how to manipulate it can access it
If Microsoft gets this implemented successfully, the thing known as privacy may as well not exist. They sre the default on most devices, so basically anyone wanting a computer (or maybe even just using one at work) will have anything they typed sold off to people who will then be able to scam them out of more money or just ruin their lives for the hell of it.
I don't think it's a question anymore. Human vs AI stories knew the problem, but didn't know the catalyst.
Here it fucking is.
If we let this slide, might as well call the idea of self-worth or individuality non-existant. Every asset of you will become another number to be sold off.
Capitalism moving on from using property/liquidity as an asset to using *existences* as an asset
😏👌 great
@GossiTheDog That doesn't even seem to be "AI" really. OCR algorithms use neural networks, but its not what Microsoft advertises. And I guess searching the indexed screenshots is also not done by an AI, but a traditional search engine...
Why??
@GossiTheDog Thinking about how simple all this works, I am really waiting for someone implementing a similar feature but for #Linux Desktops xD
All the needed components have open source projects implementing them.
Kevin Beaumont
Ryan Rester
Marcus Adams
Gavin Jones 
Thibault D.
Matti Järvinen
Scott Hanselman 👸🏽🐝🌮
Tane Piper ⁂
Andrew Prillaman
Alan Miller 
🇺🇦
Gaëtan Perrault
your auntifa liza 🇵🇷 🦛 🦦
Claudestrophobic
Jonathan Hendry
Passenger
Petrus Hilarius
Andres
Michael T Babcock
Gerardo Lisboa
THIS ACCOUNT HAS MOVED
PG&E Delenda Est
Renard 
Darkstar
Xyleth
Phel
Ash
Viraptor
Sevoris
sophia disorder
Taggart 
uzayran
thepwnicorn
FreediverX
go follow @
artisanrox
SomeGadgetGuy
Mystery Babylon 
mark
𝖋𝖑𝖔𝖗𝖎𝖓 🇨🇦
Jules Enriquez
Orc
Andrew