@paninid @astro_jcm @rysiek I already run local LLMs and am working on interfacing Audio-to-text ;) This will soon be here. Free. Opensource.
Will not likely be me that manages to put something together - but all the tech is here and omg we are many that need to just get the actual content out of these 8 min+ (needed for monetization) Youtube videos ...
@troed @paninid @astro_jcm @rysiek
Any advice or resources on building local LLMs? I'm working on a personal project of a local LLM and am always looking for ideas.
@faberfedor @paninid @astro_jcm @rysiek Well building if you mean training from scratch is way out of consumer hw league still, but just using one of the LLMs through Oobabooga works fine. I have a 12GB VRAM GPU and have used various GPTQ 13b models.
I run a local chatbot for the family matrix channel that way: https://blog.troed.se/2023/03/19/create-your-own-locally-hosted-family-ai-assistant/
What I'm working on now is which audio2text product to interface this with. For text2audio I already use Tortoise.TTS.
So the thinking here is that when I have the audio2text up I can then pipe that through a model with a large enough context window and simply ask the LLM for a summary. In chat mode, this could then also be probed with detailed questions (keeping the same context).
I've had realtime usecases in mind mostly, but that's not needed here, which means there are probably quite a few audio2text projects to go back to.
> one of the LLMs through Oobabooga
That's a(nother) new one to me.
My idea is to train in the cloud, fine-tune it (locally?) and host it locally. The fine-tuning is going to be more personal data: notes, tweets, emails, etc. How to do updates, OTOH...<shrug>
ATM I'm building an MLOps pipeline just 'cuz. I hadn't thought about an agent UI since my initial goal didn't require a UI.
Thanks for the blog post. It'll give me something to do while my bread bakes. 🙂
Thanks for that. Looks like a decent way to differentiate customers for my freelance business.
@BlinkPopShift @astro_jcm I suspect you're right, but "easier for the teacher" only matters when we overburden instructors to the Nth degree.
Redundancy of information across stylistic platforms - making the data available in "translations" for various learning strengths, is really necessary for a robustly educated, diverse populace.
Showing how is good. I can show you how an internal combustion
engine works, but this is of limited value when studying the chemistry of it. Words have value.
@BlinkPopShift @astro_jcm Easy for them; useless for the audience.
Who wants to constantly try to find the exact second on a video where the line of code you need to laboriously copy off the screen is shown? Who benefits from seeing a video shown linearly, one moment a time, vs. a page of text you can see all at once and skim visually to find the precise paragraph you need? Even being able to place markers at time points on a video doesn't help much, because you go to the marker, then wait 20 seconds for them to finish talking, then try to hold in your mind the exact, precise, thing they said, because programming isn't about getting the gist of a thing -- it's about an exact sequence of characters.
@Nefex @astro_jcm I *loathe* click-baity video titles >.<
A youtuber had a video that claimed to be on figuring out what the right yarn for a crochet bag was.
Not so. It was a video where she crocheted a bag using a specific yarn >.<
What I hate is how the "help" video is used for advertising, before, during, & after.
Surveillance capitalism.
The "advertisers" in Russia now know the make & model of your computer, operating system, software version, age, and that you're a novice in a particular area.
The perfect target for a scam, fraud, & microtargeted spam.
YEEEESSSS!
@astro_jcm but how will you learn the content without the memes, a word from our sponsor, and of course you need to like and subscribe and slap that notification bell to get spicy takes like this in your inbox daily.
For real though, I’ve found that copying the video transcript into an ai summarizer works ok most of the time.
@astro_jcm I am Autistic and I have visual processing problems, or, as I like to call it, "Ooh! It's a picture of a blue! No, there's white specks... Oh! A red patch! Oh it's ... the SpaceX rocket, pre-deconstruction... of course.". ... or, A boat on the ocean, or Spiderman. See the problem?
Now, imagine that, but the pictures MOVE! *sigh*
*headtodesk*
@astro_jcm Yep, most of the time, some text with pictures will do the job better and faster than a video.
Only a few tasks I need to learn actually require me to be shown the process step by step in video form. In those cases, a video is helpful, but they're not the majority.
@astro_jcm how the hell am I going to copy-paste from a friggin YouTube video? Was that a lower-case L or a number 1? Your screen recorder is potato so I can't say for sure.
It's almost as bad as "Quick and Easy Recipe for Falafel: Once upon a time in a land far far away, my aunt's cousin's neighbor's best friend's dog-walker's roommate's great-great-great-great-great-great-great-grandmother had a cow..."
@astro_jcm omg Yes! Why do they *do* that...?
Does anyone actually find them helpful?
Aaargg
"blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah ...,
Let's get started!
Blah blah..."