Một người dùng đang tìm kiếm model AI địa phương chất lượng cao cho nội dung NSFW, chạy trên 4060 Ti (8GB VRAM) và 32GB RAM qua Oobabooga. Các model đã thử hoặc quá chậm (Llama_3.x_70b) hoặc chất lượng chưa đạt yêu cầu của DeepSeek nhưng không bị kiểm duyệt. Ai có gợi ý model nào phù hợp không?
#AILocal #NSFWModel #4060Ti #Oobabooga #LLM #AIViệtNam

https://www.reddit.com/r/LocalLLaMA/comments/1nyqrjd/looking_for_a_highquality_local_nsfw_model_4060/

Have updated my local #LLM server from #Mistral 8B to #Qwen3 30B A3B. Still very fast. "Thinking" model so it tends to follow instructions and handle prompts with copious amounts of data better (#HomeAssistant reports).

This is all done on CPU, too. 20 cores, but still. If you have #HomeLab equipment needing a use, I highly recommend setting up #oobabooga and slapping one of these bad bois in there so you can call it via API in your other projects.

Are there any ‘multimodal’ #AI frontends for #SelfHosting out there? Currently I use #oobabooga, which is really cool but it’s text-only.

I mean like something similar to the new GPT-4o, which lets you upload pdfs, photos, access websites and etc on the chat.

Also, would the models need to be prepared for this or are the same models usable?

In this guide I explain how to use SillyTavern on Arch Linux locally: https://spacebums.co.uk/sillytavern/

#sillytavern #AI #archlinux #guide #oobabooga #roleplay #linux

Hey fellow #AI / #LLM #nerds, I've discovered a weird issue with some LLMs I've been foolin' around with. I'm using #oobabooga, and Some models seem to just quit when formatting block text.

```markdown
```

For example, the above is what *some* models do. Others don't seem to care. Something about the ``` token makes it give up. Seen anything like this?

What should one consider when choosing a model in #Huggingface?

Just the number of parameters and the filesize (the bigger these two are the better the model is)?

Which would be the best model in there? I tried running some models locally but they’re so much worse (“dumber”) than ChatGPT, I wanted to run something closer to it. Maybe the models I chose just weren’t big enough.

What is the “smarter”/biggest model in there?

I have an empty ssd I could use if the model is really big, and I have a good gpu so that’s not a problem either as long as the model doesn’t need like 10 gpus to run

Edit: okay apparently some models need like 300GB of VRAM to run, so lemme ask this differently:

What is the biggest, best model I can run on a RTX 4090? (24GB VRAM)

(64GB RAM)

Edit 2: apparently there’s this bitsandbytes thing that could help?

(Probably irrelevant but I’ll use the #oobabooga webUI)

#AI #LLM #GPT #GPT4all

I went to try Mixtral 8x7b on my gpus 4 x 24GB Telsa p40's

https://mistral.ai/news/mixtral-of-experts/

I loads the whole thing to my GPUs, all ready everything set,
type "hello AI how are you?" into ooba booga text generator
click SEND

**screens go black
**computer starts beeping like cray cray

⠀⠀⠀⠀⠀⠀⠀⣀⣤⣶⡶⠶⠶⠦⣄⠀⠀
⠀⠀⠀⠀⠀⢠⣾⡿⠋⠁⠀⠀⠀⠀⡨⠂⠀⠀⠀⠀
⠀⠀⠀⠀⢀⣾⣿⣧⠀⢠⠔⣐⡀⠈⠶⠶⠄⠀⠀⠀
⠀⠀⠀⠀⢸⣿⣿⣿⡄⠀⠈⠙⠁⡆⠑⢠⠈⡀⠀⠀
⠀⠀⠀⠀⠈⢿⣿⣍⠃⠀⠀⠀⣀⠀⠁⡀⠁⣇⣀⣀
⠀⣠⣤⣶⣾⣿⣿⣿⣭⠀⠀⠀⠀⠈⠉⠁⠀⣿⣿⣿
⠿⠿⠿⠿⠿⠿⠿⠿⠖⠌⠐⠂⠀⢀⡀⠠⠿⠿⠿⠿

#LLM #AI #oobabooga #Mixtral

LLM for GTX1080 (8GB) for local use.

https://lemmy.dbzer0.com/post/7095567