John U. Balis

@johnubalis
65 Followers
350 Following
63 Posts

Open source AI enthusiast.

Maintainer of several generative AI open source projects on Github.

I post about AI and open source.

Bloghttps://balisujohn.github.io/
Githubhttps://github.com/balisujohn

I'm pleased to announce localwriter, a libreoffice writer extension that allows local-inference language model editing using text-generation-webui as a backend.

Repository:
https://github.com/balisujohn/localwriter
Demo:
https://www.youtube.com/watch?v=rassvcjpTA0

#opensource #libreoffice #LibreOfficeWriter #localllama #generativeAI

GitHub - balisujohn/localwriter: A LibreOffice Writer extension that adds local-inference generative AI features.

A LibreOffice Writer extension that adds local-inference generative AI features. - balisujohn/localwriter

GitHub

If anyone is interested in converting PyTorch models to GGML, I have a new blog post on the subject, complete with example code.

https://balisujohn.github.io/converting-pytorch-to-ggml/

#PyTorch #ggml #opensource

How to convert PyTorch Models to GGML for Backend Agnostic Local Inference

Guide for converting PyTorch models to GGML

jbalis webpage

If you want to get GGML and C++ experience and work on a (hopefully) impactful FOSS project, feel free to join the tortoise.cpp discord. I have tasks to work on for all skill levels. If you see this after the link is expired you can also DM for an invite.

https://discord.gg/YtCfVu5r

Join the tortoise-tts reverse engineering Discord Server!

Check out the tortoise-tts reverse engineering community on Discord - hang out with 4 other members and enjoy free voice and text chat.

Discord

tortoise.cpp (tortoise-tts ggml rewrite) is looking for open source contributors! Feel free to DM if you are interested. All C++ skill levels welcome.

https://github.com/balisujohn/tortoise.cpp

GitHub - balisujohn/tortoise.cpp: A ggml (C++) re-implementation of tortoise-tts. Under construction and seeking contributors.

A ggml (C++) re-implementation of tortoise-tts. Under construction and seeking contributors. - balisujohn/tortoise.cpp

GitHub

If anyone is looking to work on an #opensource #AI (text to speech) project and have some C++ experience, here is a great first task to work on!

https://github.com/balisujohn/tortoise.cpp/issues/2

Make the tokenizer match the tortoise-tts Tokenizer exactly · Issue #2 · balisujohn/tortoise.cpp

If people are interested in contributing to tortoise.cpp, a great first task would be getting the tokenizer to always match the tokenization tortoise-tts uses. The tokenizer I'm using in tortoise.c...

GitHub

Does anyone know of a Linux app that will let me create forms in non-form PDFs? Because occasionally I get PDFs that definitely should be form-fillable but aren't

I am specifically talking about forms, not freeform editing/annotations.

I keep being pleasantly surprised by how frequently just reporting an open source software bug or packaging issue with specific examples -- often not much more than I needed to do to figure out that it wasn't *me* causing the problem -- results in the thing being fixed.

Honestly, given how effective it is (and how good it feels!) I'm kind of shocked that more people don't report issues. Like how am I the first one to report this when the package has 1,000,000+ downloads?

Turns out you on a #Librem5 you can install android in a container and it runs at native speed. The android window shows as an app, so I can just use an android app if I find a native version lacking, neat!

If anyone is looking for low-stress opportunities to contribute to open source, getting minecraft-style automatic chicken farms to work in #minetest #mineclone2 seems to be an unsolved problem that will need changes to mineclone2's mechanics to solve. Definitely a cool opportunity to get your feet wet with Lua programming and improve a free and open source video game.

https://git.minetest.land/MineClone2/MineClone2/issues/2337

https://git.minetest.land/MineClone2/MineClone2/issues/3707

Farms Issue

An Issue to collect Farm designs that do not (or do) work to analyze why this is so. #### Simple fried chicken farm ![](https://mister-muffin.de/p/n5o6.png) The chicken on top are supposed to lay eggs which go to the dispenser which then dispenses the eggs in the second chamber below the lava where the chicks spawn into the slab. Once they grow up they're supposed to be killed and fried by the lava dropping their cooked corpse into the hopper below later to be consumed by a hungry player. This currently does not work because: * Comparators don't seem to correctly reflect the dispenser state. * The eggs don't fall through the slab into the hopper. * The dispenser does not dispense anything with a slab in front.

Mesehub

@johnubalis 12 gigs, although it's only using ~8gigs of that for inference.

The backend is based on https://github.com/ggerganov/llama.cpp which can combine both cpu and graphics card inference

Right now, because I have other things running that need the GPU I only have 30/43 layers offloaded to the GPU, the rest are CPU.

I found 13b was the right balance speed/usefulness. But with this setup I can also run quantized 34b model with some gpu offloading at a few tokens per second.

GitHub - ggerganov/llama.cpp: LLM inference in C/C++

LLM inference in C/C++. Contribute to ggerganov/llama.cpp development by creating an account on GitHub.

GitHub