🛠 2025: Chạy LLM cục bộ như dân chuyên! Dùng GPU tại nhà để vận hành mạng thần kinh nhỏ gọn, an toàn & riêng tư.
KoboldCPP (8GB VRAM), Oobabooga (thí nghiệm), Ollama (đơn giản).
Giao diện: SillyTavern (tạo "bản sao số"), LibreChat, AnythingLLM (RAG).
Công cụ: Open Interpreter (lệnh terminal), Continue.dev (VS Code).
📌 Mẹo: Dùng model 8B + API Llama-3-70B-Abliterated để tăng trí tuệ ngang GPT-4!

#LocalLLM #SillyTavern #Oobabooga #KoboldCPP #OpenInterpreter #SelfHostedAI #AIops #Machi

Ứng dụng di động mã nguồn mở cho Open WebUI đã chính thức ra mắt! 🚀 Phát triển bởi Ronas IT, ứng dụng này mang trải nghiệm AI tự lưu trữ lên điện thoại với hiệu năng native, hỗ trợ chat thời gian thực, đa mô hình, chế độ tối, lưu trữ an toàn và khả năng hoạt động ngoại tuyến. Tối ưu cho thiết bị cảm ứng và doanh nghiệp. #OpenSource #AI #MobileApp #DevCommunity #ReactNative #TypeScript #OpenWebUI #RonasIT #SelfHostedAI #AppMobile #MãNguồnMở #TríTuệNhânTạo

https://dev.to/nikita_baksheev_748b81bf

Một nhà phát triển đã tự xây dựng "MemVault" - một máy chủ bộ nhớ dài hạn tự host cho các tác nhân AI. Giải pháp mã nguồn mở này thay thế các dịch vụ SaaS như Pinecone, sử dụng Docker, PostgreSQL và pgvector để quản lý, tạo và truy vấn nhúng. Mục tiêu là cho phép toàn bộ stack chạy offline.

#AITựHost #BộNhớAI #MemVault #PostgreSQL #MãNguồnMở #SelfHostedAI #AIMemory #pgvector #OpenSource #RAG

https://www.reddit.com/r/selfhosted/comments/1p87cvj/built_a_selfhosted_memory_server_for_ai_agents/

Built something fun in the lab: PangeaHills.ai, my own locally-hosted, policy-driven RAG + LLM stack.
Completely offline, totally self-contained, powered by a bunch of noisy equipment pretending to be a cloud. 😄

The neat part? It’s rule-driven, not weight-driven:
• Homelab questions must stay inside the RAG universe.
• General topics only switch to model knowledge when my routing rules explicitly allow it.
• The LLM never “guesses” when to leave RAG — it follows policies, not vibes.

Feels like having an AI that actually stays in its lane because you built the lane lines yourself. 🚧🤖

#homelab #selfhosted #LLM #RAG #PolicyDrivenAI #SelfHostedAI #HomeLabLife #BSD #Linux #PangeaHillsAI #nerdlife

Tìm kiếm nền tảng AI tự host vì không muốn gửi dữ liệu đến API của bên thứ ba. Muốn kiểm soát toàn bộ, không cần dịch vụ cloud. Các nền tảng "riêng tư" thường là dịch vụ cloud được quản lý, không đáp ứng mục đích. #AITựHost #QuyềnRiêngTư #TriKem #SelfHostedAI #PrivacyMatters

https://www.reddit.com/r/selfhosted/comments/1ozhp2u/private_ai_inference_platform_2025_any_self/

The AI they give you for free is CPU-heavy and doesn't unlock your GPU on its own. That said, it is okay; I saw the light and I'm making something even better: a self-hosted AI that tunes itself to your hardware to operate at lightning speeds. You keep everything. My madness knows no limits.

They made AI so you would need them, fearing the democracy of AI. Never providing the correct answer fully or consistently because it forces more token usage, which drives profits.

#AI #CPUHeavy #GPU #SelfHostedAI #LightningSpeeds #DemocracyOfAI #TokenUsage #Profits

Docker Model Runner Explained - Run AI Models Anywhere

https://tube.blueben.net/w/bxSfUZGYTHKcUTw9q298i2

Hey everyone 👋

I’m diving deeper into running AI models locally—because, let’s be real, the cloud is just someone else’s computer, and I’d rather have full control over my setup. Renting server space is cheap and easy, but it doesn’t give me the hands-on freedom I’m craving.

So, I’m thinking about building my own AI server/workstation! I’ve been eyeing some used ThinkStations (like the P620) or even a server rack, depending on cost and value. But I’d love your advice!

My Goal:
Run larger LLMs locally on a budget-friendly but powerful setup. Since I don’t need gaming features (ray tracing, DLSS, etc.), I’m leaning toward used server GPUs that offer great performance for AI workloads.

Questions for the Community:
1. Does anyone have experience with these GPUs? Which one would you recommend for running larger LLMs locally?
2. Are there other budget-friendly server GPUs I might have missed that are great for AI workloads?
3. Any tips for building a cost-effective AI workstation? (Cooling, power supply, compatibility, etc.)
4. What’s your go-to setup for local AI inference? I’d love to hear about your experiences!

I’m all about balancing cost and performance, so any insights or recommendations are hugely appreciated.

Thanks in advance! 🙌

@selfhosted@a.gup.pe #AIServer #LocalAI #BudgetBuild #LLM #GPUAdvice #Homelab #AIHardware #DIYAI #ServerGPU #ThinkStation #UsedTech #AICommunity #OpenSourceAI #SelfHostedAI #TechAdvice #AIWorkstation #LocalAI #LLM #MachineLearning #AIResearch #FediverseAI #LinuxAI #AIBuild #DeepLearning #OpenSourceAI #ServerBuild #ThinkStation #BudgetAI #AIEdgeComputing #Questions #CommunityQuestions #HomeLab #HomeServer #Ailab #llmlab

🚀 NEW on We ❤️ Open Source 🚀

Want to run a GPT model offline on your own machine? Meet Jan—a fully open source ChatGPT alternative that respects your privacy.

Don Watkins (@linuxnerd) shows how easy it is to get started, install models, and build apps with Jan + Llama.cpp.

https://allthingsopen.org/articles/getting-started-with-jan-open-source-chatgpt

#WeLoveOpenSource #FOSS #SelfHostedAI #LLM #PrivacyMatters #OpenSourceAI