first_last_whateverSOTA LLMs via API - EU hosted - What do you use?
There’s Mistral AI in France. And Microsoft offers the OpenAI/ChatGPT models running in EU datacenters as part of Azure.
Sadly Mistral’s models are somewhat behind at the moment from the benchmarks I’ve been looking at. But yeah that would probably be the most european solution.
Microsoft hosting in EU datacenters doesn’t help with Microsoft being a US corporation. Therefor they need to hand data even from EU servers to US government by law (CLOUD Act). As far as I know they don’t even claim differently.
As for now I still avoid setting it up myself, because it is just much more expensive if you don’t run very high loads. The per token price for occasional or low volume load is just better using API services.
Hmmh. I think you already found the well-known services. I found one more: regolo.ai from Seeweb?!
But I’m not aware of anything that properly fits what you’re asking for.
I’ll mostly tinker around with RunPod instances. But that’s notwhat you’re looking for, either.
I use Mammouth.ai. It similar to Regolo as that it is a privacy friendly interface between you and a service via their API (for instance Mistral, Gemini, OpenAI, etc.). Not as good as running your own, but the next best thing IMHO. What I like about Mammouth is that you have access to many different services for a standard monthly fee.
Daniel Gerdesmann@fediuser76 @first_last_whatever Nice, did not know them. Alternatives with no flat fees per month are Requesty AI (UK, but with EU focus), and Eden AI (french). Here you pay around 5% on top of model provider API prices. Probably a bit cheaper for low volume or irregular usage. Provider and model coverage is extensive. Most other European API router services I found are enterprise-oriented.
@fediuser76 @first_last_whatever I should mention that these are API only, without a Chat interface. But they work with self-hosted OpenWebUI and LibreChat instances.