wrote a blog post trying to understand how local LLMs work so I can (in part 2) run a couple and squeeze as much performance out of them as possible on meager hardware
it's 1800 words, no pictures, very boring, i dont blame you if you dont read it, i wrote it mostly for my own amusement
@decryption Yeah definitely useful, I had a general understanding of how they work, but your write-up connected a few dots for me.
That Ryzen 395 is a little beast, but honestly I was expecting more than $3.5k for 128gb of RAM.
Look forward to part 2!
@wallamba they seem to be in the $4500-ish range but this one is an outlier
decryption
Wallamba