There *has* been a lot of talk about the problems with so-called "AI" but one I don't feel gets enough attention is that "AI" products are surveillance products. "AI" is inevitably run in a cloud service, and in order for the AI to know what to generate some amount of the context within your application— usually it's not clear to the user what context, or how much— has to get sent to the cloud. The more of my local app state that gets transmitted over the Internet, the less comfortable I am.
So consider the "Copilot button". I cannot imagine a way this could get implemented that doesn't come down to "there's a trap button on your keyboard that every time you press it, some nonobvious chunk of local/personal information gets sent over the Internet and bounces between multiple corporations". The privacy policy will claim the information is not "retained", but the moment this centralized data pipe exists every intelligence service on earth will have a high incentive to get a tap on it.
@mcc I mean, we already had that. In the late 1990s/early 2000s most mobile phones had a WAP button that would start mobile internet at horrendous rates and whenever you pushed it accidentally you would scramble to abort abort abort cancel abort as fast as you could before you were charged like 5 bucks for opening the home page.