RomanMar 4

Can anyone recommend a privacy friendly SaaS #llm #inference provider? It needs to support *function calling* on at least one of the more recent #openweights models:

- gpt-oss
- Olma3
- Apertus? (I did not yet succeed using it)

There should be some level of cost control. Ideally a hourly rate limit. European solutions are preferred.

Use case is to have a fallback for demos or experiments where local inference is not practical. Monthly costs should go towards 0 when not used.

#selfhosting

Show threadfcat
@guesser In general, privacy != SaaS
Mar 6 at 1:44pmWeb
Show threadRomanMar 6

@f As a rule of thump, I agree. But in the case of someone who can not or does not want to run things on self-owned hardware:

**What is the next best solution when it comes to running LLM queries?** Renting a GPU is hardly practical for most users.

I should have been more clear. The project I am working on clearly advocates to run the LLM inference locally. It would be nice to offer an alternative.