Can anyone recommend a privacy friendly SaaS #llm #inference provider? It needs to support *function calling* on at least one of the more recent #openweights models:
- gpt-oss
- Olma3
- Apertus? (I did not yet succeed using it)
There should be some level of cost control. Ideally a hourly rate limit. European solutions are preferred.
Use case is to have a fallback for demos or experiments where local inference is not practical. Monthly costs should go towards 0 when not used.
Privacy friendly, you're looking for a local model.
Not hard to do with a LLama or a Mistral.
Nice to have big GPU (VRAM)
@n_dimension Yes, running the model directly is the desired use case for the selfhosted assistant I am creating (https://gitlab.com/RmMsr/relagent). And that works great.
To let people try it or if someone has no other option, I want to suggest something more suitable than just chatgpt, gemini, openrouter or whatever turns up on the internet that day. ;-)
Just occured to me...
...check out one of my stickies.
I have back of a napkin concept for a small #community #ai
Roman
Wulfy—Speaker to the machines