David Gerard4d ago

EchoLeak — send an email, extract secret info from Microsoft Office 365 Copilot AI

https://pivot-to-ai.com/2025/06/15/echoleak-send-an-email-extract-secret-info-from-microsoft-office-365-copilot-ai/ - text
https://www.youtube.com/watch?v=1wfyJd6bekg&list=UU9rJrMVgcXTfa8xuMnbhAEA - video

EchoLeak — send an email, extract secret info from Microsoft Office 365 Copilot AI

You can’t put data into an LLM and then filter the output to block some of it. Once it’s in the model, it’s in the model. If you put confidential business information into Microsoft 365 Copilot, an…

Pivot to AI
Show threadDan Sugalski3d ago
@davidgerard the amount of enthusiastic idiocy involved with all these LLM business integrations is really astonishing, even for someone as cynical about the capabilities of upper management as I am.
Show threadJeremy Kahn3d ago

@wordshaper

It really is gobsmacking

If I or any other actual (white collar) worker, no matter how senior, were to suggest using a technology that so clearly *doesn't work* and is so *eye-wateringly expensive* we would be laughed at or summaily fired

Ed Zitron's phrase The Business Idiot seems more and more apt (tho I wish there were a less ableist term)

@davidgerard

Show threadDan Sugalski3d ago
@trochee @davidgerard frankly I am *very* glad to be eyeing retirement and will be bailing out before I’m forced to use this stuff. Its value is so hugely negative it boggles the mind.
Show threadOzzelot  3d ago
@wordshaper @trochee @davidgerard I still naively hope it will fall apart/into obscurity within the next few years. If not... well, interesting times ahead.
Show threadPaul Walker
@ozzelot @wordshaper @trochee @davidgerard Not sure I’m going to get to retirement before this falls apart. If not, I’m banking on the “making money fixing the mess” approach.
Jun 16 at 10:11amWeb
Show threadOzzelot  3d ago
@arafel
A fine approach if you can take it. I'm banking on the "getting into an entirely different line of work somehow and laugh at mainstream IT from a distance" approach. (Still not sure what that line will be.)
@wordshaper @trochee @davidgerard
Show threadPaul Walker3d ago
@ozzelot @wordshaper @trochee @davidgerard That's also on the table. I'm still in the "watching with quiet horror as people apparently lose their minds" stage.