OnPrem.LLM is a very good use of your LLMs running locally. https://amaiya.github.io/onprem/
OnPrem.LLM – onprem

A tool for running on-premises large language models on non-public data

onprem