Mastodawn

Any alternatives to GPT4All for local ChatGPT-like experience on Linux?

https://lemmy.ml/post/30370832

Any alternatives to GPT4All for local ChatGPT-like experience on Linux? - Lemmy

I don’t have many specific requirements, and GPT4All is working mostly well for me so far. That said, my latest use case for GPT4All is to help me plan a new Python-based project with examples as code snippets, and it lacks a specific quality of life feature, that is the “Copy Code” button. There is an open issue on GPT4All’s GitHub [https://github.com/nomic-ai/gpt4all/issues/2784], but as there is no guarantee that feature will ever be implemented, I thought I’d take this opportunity to explore if there are any other tools out there like GPT4All that offer a ChatGPT-like experience in the local environment. I’m neither a developer nor a sysadmin, so a lot of self hosting guides go over my head, which is what drew me to GPT4All in the first place, as it’s a very accessible to non-developers like myself. That said, I’m open to suggestions and willing to learn new skills if that’s what it takes. I’m running on Linux w/ AMD hardware: Ryzen 7 5800X3D processor + Radeon RX 6750 XT. Any suggestions? Thanks in advance!

will May 19, 2025

OpenWebUI is a superb front-end and supports just about any backend that you think of (including Ollama for locally hosted LLMs) and has some really nice features like pipelines that can extend out its functionality however you might need. Definitely has the “copy code” feature built-in and outputs markdown for regular documentation purposes.

Grimy May 19, 2025

OpenWebUI is also my go-to. It works nicely with runpods vllm template, so I can run local models but also use heavier ones at minimal cost when it suits me.

yo_scottie_oh May 20, 2025

Thanks for the tip about OpenWebUI. After watching this video about its features, I’m intrigued.

Would you mind sharing a little bit about your setup? For example, do you have a home lab or do you just run OpenWebUI w/ Ollama on a spare laptop or something? I thought I saw some documentation suggesting that this stack can be run on any system, but I’m curious how other people run it in the real world. Thanks!

Exploring Open WebUI: Features, Models, & Tools [Updated]

YouTube

will May 20, 2025

Sure, I run OpenWebUI in a docker container from my TrueNAS SCALE home server (it’s one of their standard packages, so basically a 1-click install). From there I’ve configured API use with OpenAI, Gemini, Anthropic and DeepSeek (part of my job involves evaluating the performance of these big models for various in-house tasks), along with pipelines for some of our specific workflows and MCP via mcpo.

I previously had my ollama installation in another docker container but didn’t like having a big GPU in my NAS box, so I moved it to its own box. I am mostly interested in testing small/tiny models there. I again have Ollama running in a Docker container (just the official Docker image), but this time on a Debian bare-metal server, and I configured another OpenWebUI pipeline to point to that (OpenWebUI lets you select which LLM(s) you want to use on a conversation-by-conversation basis, so there’s no problem having a bunch of them hooked up at the same time).

arcane May 21, 2025

Thank you, this is really helpful to inform my setup!

simple May 19, 2025

LM Studio although i’ve never tried the linux version.

LM Studio - Local AI on your computer

Run local AI models like gpt-oss, Llama, Gemma, Qwen, and DeepSeek privately on your computer.

LM Studio

vivendi May 20, 2025

I have. AppImage only is a weird choice but it works well

KiwiTB May 19, 2025

Page assist, runs in your browser and interfaces with ollama.

Even_Adder May 19, 2025

GitHub - LostRuins/koboldcpp: Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

Run GGUF models easily with a KoboldAI UI. One File. Zero Install. - LostRuins/koboldcpp

GitHub

Lung May 19, 2025

VSCode with the open source Cline extension. Easily the best open option, works everywhere. Excellent coding and planning agent, I use it for everything

yo_scottie_oh Jul 27, 2025

If I go the VSCode/Codium route, would I also need to look into something like Ollama to actually drive the LLMs?

Lung Jul 29, 2025

Yes Cline supports pretty much every LLM API, local model, etc. But if you’re serious about having an AI assistant that actually works, it’s unlikely you’ll be able to run it locally rn. Your best cheap/free-ish option is the Google models. The best overall is still Anthropic

heisenbug4242 May 19, 2025

If you’re looking for a web UI and a simple way to host one yourself, nothing beats the “llama.cpp” project. They include a “llama-server” program which hosts a simple web server (with a chat webapp) and OpenAI-compatible API endpoint. It now also supports multimodality (for models that support multimodality), meaning you can for example upload an image and ask the assistant to describe the image. An example command to set up such a web server would be:

$ llama-server --threads 6 -m /path/to/model.gguf

Or, for multimodality support (like asking an AI to describe an image), use:

$ llama-server --threads 6 --mmproj /path/to/model/mmproj-F16.gguf -m /path/to/model/model.gguf

brucethemoose May 20, 2025

You can squeeze a lot more performance out with a newer framework and a model tailored for your GPU and task.

I’d recommend:

Kobold.cpp rocm, follow the quick-install guide here: github.com/YellowRoseCx/koboldcpp-rocm/?tab=readm…

Download this quantization, which fits in your VRAM pool nicely and is specifically tuned for coding and planning, select it in kobold.cpp: huggingface.co/…/Qwen3-14B-Esper3.i1-IQ4_NL.gguf

Use the “corporate” UI in kobold.cpp in your browser. If that doesn’t work well, kobold.cpp also works as a generic OpenAI endpoint, which you can access from pretty much any app, like openwebui.com

GitHub - YellowRoseCx/koboldcpp-rocm: AI Inferencing at the Edge. A simple one-file way to run various GGML models with KoboldAI's UI with AMD ROCm offloading

AI Inferencing at the Edge. A simple one-file way to run various GGML models with KoboldAI's UI with AMD ROCm offloading - YellowRoseCx/koboldcpp-rocm

GitHub

Sibyls Aug 18, 2025

I will also second OpenWebUI. They’ve also recently made a lot of improvements to the system. I have a workflow for an emergency system using indexed offline zim files and exposing the index using FastAPI, then accessing it from OWI. Basically, I can run a small parameter model with the knowledge of Wikipedia or medical indexes. Perfect to run off a phone or small device in emergency situations with no power or internet.