mccThere *has* been a lot of talk about the problems with so-called "AI" but one I don't feel gets enough attention is that "AI" products are surveillance products. "AI" is inevitably run in a cloud service, and in order for the AI to know what to generate some amount of the context within your application— usually it's not clear to the user what context, or how much— has to get sent to the cloud. The more of my local app state that gets transmitted over the Internet, the less comfortable I am.
So consider the "Copilot button". I cannot imagine a way this could get implemented that doesn't come down to "there's a trap button on your keyboard that every time you press it, some nonobvious chunk of local/personal information gets sent over the Internet and bounces between multiple corporations". The privacy policy will claim the information is not "retained", but the moment this centralized data pipe exists every intelligence service on earth will have a high incentive to get a tap on it.
Henri@mcc I'm sure it will already preprocess your data before you press the button.
But otherwise, local LLMs are also a thing, but not reaching to the same level as GPT-4 and also they take a lot of resources.
@slyecho If this is the case, it just becomes a matter of reversing the "preprocessing".
In the case of Microsoft, I think we can assume that the "AI" will never be local and as much as possible will occur on the server side, because Microsoft's entire motivation in pushing "AI" this way is to generate business for Azure.
@mcc I really doubt it, because it would be cheaper to do as much as possible on the customer's machine and using their electricity. Business AI is something different.
@slyecho I'm assuming the goal is to *spend* as much money as possible, so numbers in a spreadsheet somewhere go up, so finding a cheaper way to do it doesn't help. You're right this is business irrational, so I assume either Microsoft thinks they're going to attach a revenue stream to this somehow later, or else Microsoft is doing something irrational.
@mcc It is losing them money right now, massive resource costs, because if users use it without an additional subscription, MS will pay it themselves.
But also look at all the AI accelerators AMD and Intel are already putting in their CPUs. Windows 12 gonna have like 32 GB RAM minimum or something lol. Also not impossible that there will be a subscription fee added on.
And conveniently enough, all your file are already in OneDrive and were indexed by the AI by launch, just in time to answer questions about all your juicy personal life!
Asta [AMP]@slyecho @mcc They have very little incentive to ship and run machine learning on the devices. For one, as MCC mentioned numerous times, the whole point is bolstering their Azure business, and running LLMs locally defeats that.
In addition, they don't care about non-OpenAI developed models, and the OpenAI ones are gigantic pieces of garbage that can't be run locally like other LLMs for a variety of reasons. They don't WANT to ship the model to you, even if they could (they can't). But also, they don't want you to disconnect from their servers.
But also, finally, even if they did ship LLMs capable of ... I don't want to say performing, but I guess capable of doing the things they want them to, it's doubtful most people's low end devices could run them. And also, they'd have to spend time and money training those local models to act like CoPilot, which they're not going to do because they just spent 10 billion + helped engage in a reverse-coup on/at OpenAI.
Until they see blowblack from everyone realizing these models are useless and they have to come down off their coke high and undo a lot of this garbage, I'm sure they'll keep pushing this off-site angle... I doubt they want to give up that juicy data stream.
In addition, they don't care about non-OpenAI developed models, and the OpenAI ones are gigantic pieces of garbage that can't be run locally like other LLMs for a variety of reasons. They don't WANT to ship the model to you, even if they could (they can't). But also, they don't want you to disconnect from their servers.
But also, finally, even if they did ship LLMs capable of ... I don't want to say performing, but I guess capable of doing the things they want them to, it's doubtful most people's low end devices could run them. And also, they'd have to spend time and money training those local models to act like CoPilot, which they're not going to do because they just spent 10 billion + helped engage in a reverse-coup on/at OpenAI.
Until they see blowblack from everyone realizing these models are useless and they have to come down off their coke high and undo a lot of this garbage, I'm sure they'll keep pushing this off-site angle... I doubt they want to give up that juicy data stream.
@aud @mcc "bolstering their Azure business" just doesn't make sense when they are the Azure customer. Well, what makes sense is that the cost for themselves is lower than for third-parties.
What they are doing now is trying to race ahead of the competition and hope that the customer base will get hooked on the tech. Monetization comes later.
But still there is the fact that CPU manufacturers are putting NPUs into their new products means that there is a demand for them coming from somewhere. The models that run locally may not be the big ones that run in Azure but they may be part of some kind of system that will take advantage of it. I think maybe speech recognition, translation, speech synthesis running locally for lower latency could be one of those.
@slyecho @mcc Don't get me wrong; I don't think you're wrong. I just don't think Microsoft, right now, is likely interested in doing any of that. Not that I don't think it's possible it won't come down the line.
These companies do an insane amount of weird self dealing when it clouds to their cloud compute as it justifies their investments and then they can say "wow, azure is doing great!" even if it's like, you know, costing every other department money. Plus, with 10 billion invested in OpenAI, and OpenAI not having any downloadable models, it helps bolster their own revenue chart.
(it's also worth pointing out OpenAI is almost certainly going to roll out larger fees and, I assume, doesn't want to open up their models for fear of helping all the pending lawsuits against them).
But these companies infamously charge market rate to their own internal divisions; pretty sure when groups inside the various cloud companies use compute, they have to spend the same amount as other people. It's totally weird bullshit.
These companies do an insane amount of weird self dealing when it clouds to their cloud compute as it justifies their investments and then they can say "wow, azure is doing great!" even if it's like, you know, costing every other department money. Plus, with 10 billion invested in OpenAI, and OpenAI not having any downloadable models, it helps bolster their own revenue chart.
(it's also worth pointing out OpenAI is almost certainly going to roll out larger fees and, I assume, doesn't want to open up their models for fear of helping all the pending lawsuits against them).
But these companies infamously charge market rate to their own internal divisions; pretty sure when groups inside the various cloud companies use compute, they have to spend the same amount as other people. It's totally weird bullshit.
@slyecho @mcc part of me thinks/suspects? that their justification for doing so is two fold: one, it makes the cloud compute division look more profitable; two, it incentivizes their own employees not to be "wasteful"; and maybe as a distant third, it gives the appearance of being "fair" to any regulators who might sniff around.
@aud Certainly seems crazy and kind of a gamble from MS's part.
@slyecho I agree. They're not immune to jumping on trends, though, for perceived advantage (which you brought up and is DEFINITELY another reason they're pushing so hard); they already ignored and laid off the groups that warned them that this technology was nowhere near ready for this kind of deployment, so.