"...a would-be hacker would need to gain physical access to your device, unlock it and sign in before they could access saved screenshots."

I've got some news for Microsoft about how domestic abuse works.

https://www.bbc.com/news/articles/cpwwqp6nx14o

Microsoft Copilot+ Recall feature 'privacy nightmare' - BBC News

The ICO wants to know the safeguards around Recall, which can take screengrabs of your screen every few seconds.

BBC News
@evacide Or social engineering of any kind?

@evacide

... as opposed to all the would-be hackers who have never thought to try to unlock a device and sign into it, or access data without proper credentials.

It's like Microsoft is just sort of taunting hackers to try and get it broken as quickly as possible for some reason. Is this feature being implemented because somebody lost a bet, or the NSA has compromat on Nadella, or what?

@wrosecrans This feature is being implemented because there were zero survivors of domestic abuse involved the high-level decision-making.

@evacide I absolutely believe you there. But I still struggle to understand why it got implemented. There are a zillion other obvious reasons it's a bad feature that one would notice even if they weren't sensitive to that specific issue.

This is gonna have screenshots of HIPAA protected data. Trade secrets. API keys. Passwords. HR department PII. GDPR protected stuff. On and on and on.

@wrosecrans @evacide Nobody consulted a policy and compliance specialist about this. It’s shocking that Microsoft didn’t get input from at least one. This would violate a lot of data protection policies for many enterprise customers.
@MisuseCase If I had to guess, the feature is not compliant with Microsoft's own legal department's retention policy, and Microsoft's lawyers are about to scream about the fact that if MS gets sued, the blast radius for document discovery just exploded if they don't disable it internally.

@wrosecrans @MisuseCase I would be extremely surprised if this doesn't ship with a GPO to disable it.

(Also, MS not enabling group policy on consumer focused windows editions probably ranks alongside the Win8 start menu destruction as one of the worst design decisions they've ever made)

@azonenberg Sure, but the biggest risk is to people and orgs that aren't executing infosec perfectly. Ooops we had a bad password policy multiplied by ooops we left Recall's GPO default.

In a hypothetical perfect IT environment where all GPO's and such are perfectly managed, Recall probably poses little risk to start with. It's only dangerous in the real world.

@wrosecrans Yeah agreed. It's just one of 500 catastrophically horrible anti features that people will need to turn off to regain some semblance of a secure baseline.
@wrosecrans @MisuseCase this was also the case for copilot which i'm pretty sure still has the CCPA violation extant among the claims in the class action suit for slurping up all code input including e.g. passwords and API keys but they thought they could get away with that via one-off modifications to hamper evidence collection so unclear why their lawyers would think this is any different

@hipsterelectron

I continue to be fucking baffled by Copilot. I assume the engineers just fully lied to the lawyers in order to get legal to sign off on it.

I can't imagine a lawyer understanding the plan and being like, yup, let's just YOLO stealing at the courts and find out what happens. Could be neat.

@wrosecrans i believe openai is being used as a front company to derisk breaking the law and they are playing a much longer game than just copyright but instead surveillance and monopoly go hand in hand https://circumstances.run/@hipsterelectron/112476914914182012
d@nny "disc@" mcClanahan (@[email protected])

a lot of people saying "wow openai is so silly to steal from scarjo she just sued disney" like copilot didn't directly contravene established precedent in contract, copyright, and privacy law--openai is a front company for microsoft to break the law. these people are not playing games.

GSV Sleeper Service
@hipsterelectron @wrosecrans It would be a massive boon to a central government to have a machine to harvest, collate, and analyze all citizen activity. Stasi would wet its pants at the thought of something like Replay.
@klausfiend @wrosecrans had someone else advance my thinking on this just a few moments ago actually and now i'm completely with you https://circumstances.run/@hipsterelectron/112482975521122360
d@nny "disc@" mcClanahan (@[email protected])

@[email protected] is wayyyy way ahead of me as usual https://monroelab.com/2024/05/21/all-roads-lead-to-surveillance-valley-on-windows-11-recall/

GSV Sleeper Service
@wrosecrans see recent Slack policy change: you've been using our software for years. We own you now. Good luck migrating to anything else, suckers.
@wrosecrans @evacide I'm pretty sure it's getting implemented because some credulously hype-tracking investors went "hey, AI makes the line go up, that means you *must* put more of that in right now, no other option exists."
@pstewart @wrosecrans @evacide Yup. It's FOMO in its saddest, most harmful form...
@evacide @wrosecrans why does it not surprise me that us #domesticviolence #warriors weren’t heard or even asked about this. 😞🧘🏻‍♀️
@nobodypsyd @evacide Because you would have said the lives of human beings are more important than the product lifecycle, and that's not what they want to hear.
@evacide @wrosecrans I have to believe that there were many, many developers at Microsoft the said this was a terrible idea - I also have to believe that they were simply ignored because this was somebody's pet project.
@evacide @wrosecrans I have a strong reason to suspect that nobody thought about it because tech bros.

@evacide
There is also the more sinister possibility that there were and they just didn't care.

I think we significantly underestimate the number of people who operate exclusively in their own self-interest (which is to say, making money first, then everything else.)
@wrosecrans

@evacide Between Apple's AirTags being very good stalker tools and Microsoft releasing Recall, are they trying to one-up each other for who can enable domestic abuse the most?
@evacide Also, parenting. Respect for young people’s privacy has always been extremely low. This will annihilate it completely.
@evacide I guess they've also never travelled internationally and had a border security agent ask to inspect their laptop. That's usually an "unlock the device or have it confiscated" situation.
@evacide Also clearing international customs and court-ordered discovery. Huh.
@evacide say it louder for those in the back!!! 😞
@evacide Yeah, it's really kind of frustrating when the retort from someone like Raymond Chen is that "But the attacker is already through the air-tight hatch by then." That doesn't mean you should make their job easy, what the heck?
@ashteranic @evacide yeah there are many security topics where that's a sound principle. the domestic abuse threat model is not one of them.
@evacide Sounds like a good time for people to start to migrate to Linux.
@evacide cishet white dudes have very different threat models than the rest of us

@evacide

Uh, "For example, users can opt out of capturing certain websites" = "please make an intentional document attesting to the complete list of websites you are most embarrassed by visiting. what could go wrong?"

@mattdm @evacide And how do I, a non-Microsoft user, opt out of any windows copilot+ owner having their recall recording me? Do I just cautionarily block all Microsoft user agents in my video calls and chat sessions?
@gabriel
Oof I hadn't thought of that side
@mattdm @evacide
@evacide Can’t believe they haven’t seen https://xkcd.com/538/ yet…
Security

xkcd
@evacide I’m not a hacker but this seems not true?
@evacide Your employer will definitely enable this.
@evacide additionally, a lot of "hacking" is social engineering and not technical engineering as well. all someone has to do is walk away near someone they trust but shouldn't. the only thing this feature seems to help is microsoft farm data.
@evacide and unlike every other feature in the history of computers, their thteat model is 100% accurate and "hackers" will respect it. PS ofc we back this all up to onedrive

@evacide

You are right to point out domestic abuse. In addition to "the stalker inside the house" this now provides a mechanism for people with the ability to access activity data even with very limited physical access to the computer.

Kind of like putting an air tag on someone's car.

I genuinely wonder about the corporate strategist that looks at the data breaches and scrutiny that Microsoft has brought on itself and then thought 'This is a good time to announce a keylogging product!'

@evacide I find it interesting that I have been reading about this on multiple pages, and there was literally no single comment (or article) that thought this "feature" was a good idea.

What sort of plank managed to get this through internal approval at MS?

@evacide Honestly I am not sure domestic abuse is relevant in this case. If your domestic partner is abusive/malicious and has access to credentials of your devices, this is pretty much game over already regardless of the presence/absence of that feature.
@evacide thinking about that time I watched emails disappear in real time (and later learned this was how certain email addresses were scraped)

or the time my spotify started playing a different song ("fuck you" by Lily Allen)

yep, no potential for abuse here at
all.

@aud ooooooof at both those :x

I hope you've managed to entirely excise whoever did that from your life!

@froztbyte in fact, yes! Thankfully ​​ Took a while but this was long ago now.

@evacide

This can only have nefarious purposes. I cant imagine any practical use for the device owner at all. I have been reading the comments and thinking about all the ways these devices would still access my image or details even if I dont use a microsoft PC (I'm downloading linux TODAY - this is too far). Other microsoft devices at my bank if I call customer services, facebook friends, mastadon users reading my posts, insurance company, local council, all using microsoft devices that could potentially have access to my data and countless others.

This will make our devices the perfect spy network, and I dont believe for one second that the screenshots will not be accessed by micrsoft down the line. This is just the first step to get us comfortable with their existence.

I work from home and hope to gawd my employers dont blindly go along with this.

@evacide Oh they need to get physical access to the device and input passwords?

Or, and stay with me on this journey, microsoft is not a godlike being that never rolls out code with bugs. And any, ANY exploit here is going to be such a ridiculously awful fail state.

@evacide I’m surprised the ceo has any clothes on in that picture. He certainly has lost his damn mind.
@evacide me thinks this thing will get dumped. it's too problematic.

@evacide I've also got bad news for how spyware works. Or ransomware. All of which can now also access a full video of everything you did the past few months (or longer), and extort money from you to not tell others.

Remember that spam email about "we saw you watching porn and will tell others unless you pay bitcoin"? That, but now they actually could.

@evacide
It's just such bullshit.

I've not bothered to read up on the announcements but so far haven't heard mention of any supposed benefits to this incredibly invasive feature and massive security and privacy risk.

It's as if they just can't be bothered pretending any more.

@evacide How long until employers say that they should have access to that data? Their equipment, their work you’re doing. How long until governments come knocking? I can’t comprehend the mindset that would build such a feature
@evacide If you think Microsoft added centralised signing and reporting to Minecraft chat to protect the user base I have a bridge to sell you. It was known that criminals were using Minecraft chat to bypass government or enforcement censors / monitoring of email / web and they came knocking at Microsoft’s door
@evacide Naturally, the only difference between a dissident and a criminal is wether you’re in a film and need to be portrayed as the good guys. Microsoft made it possible that even if you’re on a private server open only to whitelisted individuals, you can be banned from Minecraft entirely for saying something naughty, like “Taiwan” (no context needed) for example