Selfhosted LLM (ChatGPT) - Lemmy.world
I’ve recently played with the idea of self hosting a LLM. I am aware that it
will not reach GPT4 levels, but beeing free from restraining prompts with
confidential data is very nice tool for me to have. Has anyone got experience
with this? Any recommendations? I have downloaded the full Reddit dataset so I
could retrain the model on this one as selected communities provide immense
value and knowledge (hehe this is exactly what reddit, twitter etc. are trying
to avoid…)
Redoing home lab and need insights
https://lemmy.world/post/1241338
Redoing home lab and need insights - Lemmy.world
I recently switched from TrueNAS to Synology for my NAS. TrueNAS had served me
well, but I no longer had the time to manage it effectively. On this occasion, I
decided to overhaul my entire home lab, which had gotten pretty messy over the
years. As part of this overhaul, I will be discarding my old TrueNAS device due
to its high power consumption and bulkiness. I will keep a NUC and another NUC
alternative with slightly lower specs, but with 2+ LAN ports. With this
configuration, my plan is to use Proxmox on the NUC as the primary system and
use the second NUC as a backup. The backup NUC however ha a dedicated connection
via multiple LAN ports directly to the Synology NAS, so that would be ideal to
storage intensive tasks. My primary use case will be running containers and a
few VMs for services like Git, Pi hole, backup services and more. Although my
Synology NAS supports running containers and VMs, I prefer to keep things
separate. I’ve already taken care of my infrastructure needs and won’t be
hosting pfSense or similar services. Since I haven’t looked into best practices
lately, I’m very interested in learning new technologies like Ansible for
automation. I’m especially interested in understanding how to automate installs
and updates while working with containers and VMs. I am considering whether to
stay with Proxmox or go for a simpler distribution like Debian, Fedora or
others. Thanks for you insights!
