Brad Smith just said Recall was designed to be disabled by default. That is not true. Microsoft’s own documentation said it would be enabled by default - they only backtracked after outcry.
He has somehow got almost every detail about Recall wrong while testifying.
Obviously, I’ll wait to see the announcement but it sounds like they’ve finally realised they need to take the time and get the feature right (and frankly consider the target audience - most home users, it ain’t).
They should have announced this before or during the US House hearing.
Announcement is out. Good on Microsoft for finally reaching a sane conclusion.
- Recall won’t ship as a feature at launch on Copilot+ PCs any more.
- Won’t be available in Insider preview channel at launch, as it was pulled.
When it does appear in preview channels, privacy and security researchers need to keep a close eye on what Microsoft are doing with the feature.
Microsoft tried developing this feature in secret in a way which tried to avoid scrutiny. Thank you to everyone who stood up.
If anybody is wondering, Microsoft moved the announcement up as I scooped them 🤣
Thank you to everyone who helped out with this one, there was no way something that constantly OCR’d the screen being implemented so poorly was acceptable but Microsoft really, really dug their heels in.
Photographic memory of everything you’ve ever done on a computer has to be entirely optional, with risks explained and be done right.. or not at all. Accountability matters.
Microsoft, be better.
If anybody wonders if Recall classifies what porn you watch, yes. Aside from OCRing text it also classifies images in videos.
9 minute 50 second mark in this, screen is blurred for obvious reasons.
Here’s the clip translated around adult content with Microsoft Recall.
They filter search terms in English like nude - but don’t filter it in other languages.
Everything you view - including in videos - is classified and stored in the database regardless.
This is pretty good - detecting Microsoft Recall misuse for data exfil. https://youtu.be/SV9-dn-5uEY?si=jVz9sC4A2wKxeiBt
I tested this against the latest release of Recall and both TotalRecall and these detections still work.
Obviously Recall may well alter before it hits Insider preview channel, nobody needs to rush out detections yet.
Btw all through this saga, Microsoft Defender never triggered Recall specific alerts for me. Sophos did.
You've probably heard of Microsoft's new Recall feature by now. It's a info stealer's dream come true. There has been a lot of information release about how ...
Windows 11 24H2 preview release has been rereleased (but only for Copilot+ devices). It doesn’t include Recall any more.
Additionally the Copilot+ PCs now have an update which enables the other AI features. This wasn’t available until a few hours ago, hence the lack of unsupervised reviews of the devices. It means you will see those reviews drop after the devices launch tomorrow.
There’s a website which gives some insight into how the UI and marketing push for Copilot+ Recall came together. The actual video appears to have gone MIA.
I led the visualization for the Recall app launch, showcasing its capabilities on a 50-foot screen during the live public introduction by Yusuf. My UI team managed the project from start to finish, developing visuals in the final two weeks. Building on our Recall experiences from the Surface Pro, Surface Laptop, and Copilot+ PC sizzle videos, we enhanced these scenarios for the live stage production, demonstrating Recall's full potential. This dynamic presentation was a highlight, refining Recall’s story for a large audience.
.@JohnHammond’s video on Recall is great, and a lot of fun - should also stop history being rewritten on this one later.
I got ahold of what I think is the latest Microsoft Recall (Copilot+ Recall? Nobody knows the branding) build and.. well.. Total Recall still works with the smallest of tweaks to export the database, it's still accessible as a plaintext database with marketing as the security layer.
Another observation, the Recall backlog must be very large as it's just becoming a truck load of features being dumped on.
One thing MS needs to fix in Recall, before the Insider canary build hits again, is the MSRC bug bounty.
As far as I can see, if you find a critical or high in Recall it qualifies for *drumroll* $1k bounty, unless I'm misinformed.
That probably needs clarifying as nobody is going to sell photographic memory access to Windows devices to MS for that value - it's way more valuable elsewhere.
Should Microsoft Recall ever reappear I plan to keep checking how secure it is, because the next evolution of security cannot be Microsoft pouring petrol onto the infostealer fire.
Infostealer malware is swiping millions of passwords, cookies, and search histories. It’s a gold mine for hackers—and a disaster for anyone who becomes a target.
https://www.wired.com/story/infostealer-malware-password-theft/
XDA Developers, who were a good source of behind the scenes info during the Microsoft Recall saga, are saying Microsoft have kicked Recall into the long grass and they think it may never launch. https://www.xda-developers.com/thread/microsoft-wants-you-to-forget-about-copilot-recall-it-seems/
It’s been almost two months since Microsoft said it would launch for Insiders in “weeks” instead.
Microsoft now say Recall will available for Insider testing in October on select Copilot+ PCs.
As a community we’ll need to test the security implications out extensively.
Due to hardware requirements this will obviously be a problem, unless we can hack it to install on non-NPU systems again - I don’t know if that has been ‘fixed’ or not.
https://www.theverge.com/2024/8/21/24225439/microsoft-recall-windows-ai-feature-october-testing
Recall is back.
Overall the planned changes here are much more robust.
Some of the things are boomerangs - eg they said it wasn’t uninstallable weeks ago, but it is now. Also they said it wasn’t developed under Secure Future Initiative a few months ago.. but now say it was originally under SFI.
The proof is in the pudding obviously so hands on tests will be required. They’ve locked it to Copilot+ PC systems now, which will limit research.
Microsoft have recalled Recall again.
It still hasn't even made it to Insider preview yet, that's been delayed too, now in December.
Good, by the way. They should take the time to get it right. I still don't know what they were thinking when they had the CEO stand on stage and say it was launching on devices 6 months ago and would be fully secure, when they hadn't even done a basic security review of it.
https://www.theverge.com/2024/10/31/24284572/microsoft-recall-delay-december-windows-insider-testing
I'd be surprised if it is released in December btw, as Redmond is a ghost town in the office from basically now until mid January.
I guess a cynical version is they're trying to rush out the Insider preview during Christmas so nobody actually reviews it.. but, well, I don't think that would happen as it'd be another own goal. It probably needs 6 months in Insider release with a bug bounty, to avoid exploits dropping like Joker 2 at the box office on release.
In a newly released blog entitled "Windows: AI-powered, cloud-enabled, and secure", Microsoft say the business versions of Windows will ship with Recall disabled by default - IT departments will have to enable the feature before it is available.
This is a smart move and frankly it was incredible that the original idea was to ship this enabled by default in business - it was never, ever going to fly and hopefully Microsoft is rightly humbled by the experience.
Microsoft are getting positive press for calling Recall “one of the most secure experiences it has built”.
I’d point out - they haven’t provided a Preview build to Insiders still, and there’s been no externally provided build (outside of NDA), so nobody has been able to assess the security and talk about it. There’s no specific bug bounty for it either.
When they first announced Recall, they called it totally secure - which was laughably inaccurate. It feels like a lot of premature high fiving
Microsoft Recall is now available for testing.
https://www.theregister.com/2024/11/22/microsoft_recall_release/
It’s only available on Qualcomm Snapdragon-powered Copilot+ PCs. My feeling is we’re probably going to want to hook one up to the internet and hack RDP for unlimited sessions, to allow research - I’ll look into it.
I’ve been told Recall is eligible for bug bounty as part of the Insider programme. I think the process is supposed to be sandboxed so in theory (my reading) the payout limit should be $20k.
Microsoft are rolling out Recall to users in Windows Insider (testing) before a wider rollout to all compatible systems.
It's definitely one to watch (and yes, I am) from a security point of view.
I've took a look at the past year of work Microsoft has done on Recall, which is due to roll out to compatible Windows devices soon
tl;dr it's much better from a security and privacy point of view. My partner managed to hack my Recall memory in 5 minutes to browse prior Signal discussions, by guessing my Windows Hello PIN.
There's a bunch of risks people who enable it need to understand.
Tabletop scenario for you:
Employee gets into a dispute with employer, leaves, had sensitive role. Employer revokes access, devices etc. Employee had logged in via BYOD to email, IM etc.
Due to Recall, employee walks away with 6 months of screenshots of everything she's ever worked on in a text indexed form - every email, chat, document, Teams call with video snapshots, transcripts of verbal calls etc - even if they set M365 to not store documents locally.
What does the employer do now?
Signal have rolled out an update to all users that stops Microsoft Recall from capturing Signal conversations.
I’ve tested this and it works. Brilliant work by the @signalapp team. 💪
They call on Microsoft to build better, as there was no standardised way as an app developer to do this. Because Signal is open source, now app developers have a template to protect their users from Windows.
Signal Desktop now includes support for a new “Screen security” setting that is designed to help prevent your own computer from capturing screenshots of your Signal chats on Windows. This setting is automatically enabled by default in Signal Desktop on Windows 11. If you’re wondering why we’re on...
I found an interesting Microsoft Recall issue with the latest version - Recall is enabled on my PC, but the tray icon (bottom right) saying it is running is missing.
Edit: after a reboot, it's back. I'll keep an eye on it. After the latest Windows Update the UI wasn't visible, but it was still recording.
@GossiTheDog Curious why you’re leaving Microsoft Recall enabled?
I’m still trying to figure out the intended use case.
“Hey copilot, what was the plot of last night’s pornography?”
The 'use case' of Recall and Copilot is to change user behaviour.
By encouraging users to become more reliant on MS to perform basic tasks, users will lose the ability (the skills) to perform those tasks.
For example, try using the MS Outlook client on iOS or Android to review an email's headers. (MS removed that capability a long time ago.)
The ultimate aim of MS is to have as many people as possible change to a 'subscription' model where users have *zero* access to the OS or any installed app's code.
All systems will require internet access to boot up, with possible exceptions being 'Pro' or 'Enterprise' versions for use by companies in the field.
Local storage memory will be controlled by MS, and may eventually form a 'distributed' cloud.
Therefore, the 'use case' of Recall and Copilot is to benefit MS and NOT the people who use it.
*Highly* recommended.
It is *amazing* what this utility can do.
@GossiTheDog @signalapp And by using #Microsoft's own #DRM protections to do it too. THat's brilliant.
I have #Signal but don't use it (I don't know anyone else on it) - but I still pay a recurring donation monthly because THIS is the user(privacy, rights, security)-focused product management that I want to encourage. Way to go Meredith and team.
“And by using #Microsoft's own #DRM protections to do it too. THat's brilliant.”
that’s exactly what caught my eye. they didn’t have to hack anything. it’s there in Microsoft’s own APIs. they’ve turned the monster of their own creation against them.
@blogdiva @tezoatlipoca @GossiTheDog @signalapp
The obvious next step is for someone to use those selfsame APIs to create a tool which stops Recall from recording anything.
You love to see it.
@theogrin @blogdiva @GossiTheDog @signalapp
omg a browser plugin that marks any open tab as containing DRM enforced content.
edit: sadly, very very hard to do. there are no "standard" DRM protocols for html and related ascii text content.
@f4grx @tomas @GossiTheDog @signalapp So I don't know if its what #Signal app did (I don't have the spoons to go digging around their repo rn), but one way to invoke the #Windows #DRM protections on your app is to set the display affinity of your main window handle:
`SetWindowDisplayAffinity(hwnd, WDA_MONITOR)`
https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowdisplayaffinity
where `hwnd` is your main window handle and `WDA_MONITOR` sez only show on the monitor, all other purposes get no content.
@f4grx @tomas @GossiTheDog @signalapp
Sadly, only the application itself can set its own window display affinity; I know I just tried for an hour to write one - Windows User Interface Priviledge Isolation IUPI security prevents almost any attempt of one process to muck w/ the main window of another.
Otherwise that would be the coolest thing ever. run a little app that blacks out another app's window.
@GossiTheDog Using DRM for a change to work "for the user" is a very clever idea to prevent Windows Recall from making Screenshots.
Kudos @signalapp for the creative solution👏👏
Feel like this is the opening salvo in an escalating war, the same way youtube is fighting off uBlock Origin.
For example, that DRM attribute might soon be disabled for "non media containers" since it was devised to protect copyrighted works. Sure, that'd be petty. But it's Microsoft we're talking about here.
Then of course Signal posts a workaround, which Microsoft quickly--
@clumpytree @GossiTheDog @signalapp
In this context, media container refers to a window/object designed specifically to playback DRM protected media, such as a film, show, or music. It's an artificial conceit for sure, but media companies are pretty rabid about piracy soooo there you have it.
And yes, of course text can be copyrighted. Not sure how declaring a copyright on scrolling text in a chat window would work from a legal standpoint though. IANAL
@GossiTheDog @signalapp
> “Take a screenshot every few seconds” legitimately sounds like a suggestion from a low-parameter LLM that was given a prompt like “How do I add an arbitrary AI feature to my operating system as quickly as possible in order to make investors happy?”
🔥
@GossiTheDog @signalapp Fun, I wander if something similar can be done on #linux with #wayland so I dont accidentally leak all my DMs because of missclick when using #obs and #xdg_desktop_portal, maybe not to the level of application always denying capture ( #drm applications), but it would be cool to have an rejectlist in your linux #desktop to add/remove applications you explicitly dont want to be able to capture (with default values pulled from their #flatpak manifest or something)
If current #xdg specification doesnt allow that, does any of the desktops like #gnome, #kde, #cosmic or #hyperland thought about that?
@GossiTheDog @signalapp it merely prevents #Screenshots by claiming it's #DRM'd content.
It's a mere ask and #Microsoft could specifically close that #API and make it subject to contractual agreements (as they did with their #Antivirus API calls to disable #WindowsDefender!) if they decide this is against their wishes.
It also doesn't prevent the #Keylogger nor works against the known #CryptoAPI #backdoor affecting all #Browsers (except #Firefox and @torproject / #TorBrowser) which can be triggered by a single #HTTPS request.
The correct solution for #Signal would be to alert all their users and specifically block #Windows in general or at least #Windows11 simply because it is a #Govware and empirically cannot be made private or secure.
But that would require them to actually give a shit, which thed don't, cuz otherwise they would've stopped demanding #PII like a #PhoneNumber and moved out of juristiction of #CloudAct.
Since they are highly centralized.they certainly are capable to comply with "#Sanctions" (or whatever bs he'll claim!)...
Fixes a critical backdoor in Windows' CryptoAPI, which allows to unconsenting Update of CA Certificates in the background. See https://www.heise.de/ct/ausgabe/2013-17-Zweifelhafte-Updates-gefae...
@GossiTheDog @signalapp well.
until the next MS update rolls out 🤷♂️
but I'm glad somebody is TRYING at least!
and sharing the knowledge!