For those who aren’t aware, Microsoft have decided to bake essentially an infostealer into base Windows OS and enable by default.
From the Microsoft FAQ: “Note that Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers."
Info is stored locally - but rather than something like Redline stealing your local browser password vault, now they can just steal the last 3 months of everything you’ve typed and viewed in one database.
I've written up my thoughts on the Copilot Recall feature in Microsoft Copilot+ PCs
I think it will enable fraud and endanger users, and is not the sign of a company who are committed to security first.
Copilot+ Recall has been enabled by default globally in Microsoft Intune managed users, for businesses.
You need to enable DisableAIDataAnalysis to switch it off. https://learn.microsoft.com/en-us/windows/client-management/manage-recall
Two quick updates -
A) if you disallow recording of a website in Control Panel or GPO, in Chrome it is still recorded - disallow recording only works in Edge browser
B) Firefox and Tor Browser is recorded always, including in private mode - the exception is Hollywood DRM’d videos
I got ahold of the Copilot+ software.
Recall uses a bunch of services themed CAP - Core AI Platform. Enabled by default.
It spits constant screenshots (the product brands then “snapshots”, but they’re hooked screenshots) into the current user’s AppData as part of image storage.
The NPU processes them and extracts text, into a database file.
The database is SQLite, and you can access it as the user including programmatically. It 100% does not need physical access and can be stolen.
You have to admit that's quite an endorsement for SQLite. But maybe the SQLite license should be updated to deny its use in implementing a Torment Nexus.
@jonhendry @starchy @GossiTheDog
Given the politics of the SQLite project, I shudder to think what sort of thing they would allow/disallow.
@passenger @starchy @GossiTheDog
It's mostly just the one guy, isn't it?
@jonhendry @starchy @GossiTheDog
Richard Hipp, yeah. As with many projects, a lot of the grunt work of development was done by other people though.
My original comment was related to the notorious code of ethics which he got those other devs to pledge to while working on the project.
If you haven't read it, it's here:
https://sqlite.org/codeofethics.html
(Richard, if you're reading this toot, I deeply respect you as a database engineer, but also wtf?)
@Andres4NY @passenger @starchy @GossiTheDog
I mean, it's the rules for an order of monks, so in that context it makes sense there'd be a lot.
As a code of ethics for a software project... ehhhhh.
@Andres4NY @jonhendry @starchy @GossiTheDog
And none of those rules are "don't sexually harass people", despite that being the proximal reason why we're now doing codes of conduct. "Don't be a transphobe", "don't be a misogynist" and "don't be a racist" are also things I'd have thought to include.
But then, I'm not a literal saint, so what do I know?
@passenger @Andres4NY @starchy @GossiTheDog
Those probably could fit under various rules in a rather fuzzy and non-specific way.
I mean, “Do no wrong to anyone, and bear patiently wrongs done to yourself.” if diligently followed would probably cover all the things you mentioned.
Of course the problem is that the perpetrator probably doesn't think what they did WAS wrong, thus the need for specifics.
@mikebabcock @passenger It was meant as an ethical pledge, but the language is so overloaded that it reads as an introduction to a cult.
Having worked on the #ContributorConvenant v3 @ethicalsource and knowing a few profissional codes of ethic, maybe it is time I work on some like time.
@noplasticshower @GossiTheDog well https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf
The fuck-ups are bad and the incident response even worse so. Any org worth its salt would have blacklisted Microsoft and/or o365 as a vendor by now.
But that's okay, because they have a PR dept https://www.microsoft.com/en-us/security/blog/2024/05/03/security-above-all-else-expanding-microsofts-secure-future-initiative/ and people are actually eating it up.
@GossiTheDog they’re not even doing anything interesting like vectorising the scraped text for use in an on-board RAG type system? It’s just a DL OCR and plain old search?
Still can’t see why any business that is vaguely sane would allow this on their fleet, or why any personal user would either. It screams privacy accident waiting to happen.
@phel @GossiTheDog yeah, that’s been a persistent feeling for me too: ideally you should be able to get this information over accessibility APIs. And that would let you have much more granularity *and* access control.
Using OCR is a crutch admission to how badly data access in modern OS has gotten broken - that‘s what it really feels like to me.
Basically, this is a feature that:
- wastes processing power
- actively makes your personal info vunrable to theft (think of all the not-techy people who are gonna get scammed with this)
- benefeits random companies and scammers while actively hindering the user
- gives personal info to Copilot, and if it trains off user inputs like ChatGPT does, then using it literally hands your personal info, passwords, etc. to a database where anyone who knows how to manipulate it can access it
If Microsoft gets this implemented successfully, the thing known as privacy may as well not exist. They sre the default on most devices, so basically anyone wanting a computer (or maybe even just using one at work) will have anything they typed sold off to people who will then be able to scam them out of more money or just ruin their lives for the hell of it.
I don't think it's a question anymore. Human vs AI stories knew the problem, but didn't know the catalyst.
Here it fucking is.
If we let this slide, might as well call the idea of self-worth or individuality non-existant. Every asset of you will become another number to be sold off.
Capitalism moving on from using property/liquidity as an asset to using *existences* as an asset
😏👌 great
@GossiTheDog That doesn't even seem to be "AI" really. OCR algorithms use neural networks, but its not what Microsoft advertises. And I guess searching the indexed screenshots is also not done by an AI, but a traditional search engine...
Why??
Surveillance.
(I think Microsoft has not considered Windows users to be their customers for many years by now, and Silicon valley was initially funded by the US defense department - with whom MS has contracts worth billions. Even Teams is obviously tailored only for managers, you can't actually be productive with it.)
@mhoye @GossiTheDog Recall seems to be a giant data suction pump with no escape.
Use Chrome, Firefox: scrape the data via AI
Use Edge: slurp the data directly
🤮
Days until TOR project figures out how to invoke DRM API over the entire window: ___
@GossiTheDog Could you conceivably create a browser extension that just DRMs the whole thing?
/me wonders what the least expensive DRM license is
Kevin Beaumont
your auntifa liza 🇵🇷 🦛 🦦
Vanta Puce
Jonathan Hendry
Passenger
Petrus Hilarius
Andres
Michael T Babcock
Gerardo Lisboa
THIS ACCOUNT HAS MOVED
PG&E Delenda Est
Renard 
Darkstar
Xyleth
Phel
Ash
Viraptor
Sevoris
sophia disorder
Taggart 
uzayran
thepwnicorn
FreediverX
go follow @
artisanrox
SomeGadgetGuy
Mystery Babylon 
mark
Lorenz
Wil
Richard W. Woodley ELBOWS UP 🇨🇦🌹🚴♂️📷 🗺️
Sean
2xfo
Gustavo Litovsky
PhD9
mhoye
Matthew Skelton
Noah Cook
Lando
Justin Scholz
vandenberglegs
Ásthar con té 🍵⛤ (Elle/They)
EvilGinger- Queen of Naps
Simonoid
Fennix
SPAM 
r00t f0lds Team# 258829 
argonaut
TheTomas