FOSS Advent Calendar - Door 23: Generating Text with Text-Generation-WebUI
Today we explore Text-Generation-WebUI, a FOSS tool that lets us run large language models locally through a simple web interface. For this demo, I am using 20B GPT-OSS from OpenAI in 4-bit quantization.
I tested it with a physics/chemistry prompt: "Derive the general formula for the energy levels of a particle confined in a 2D infinite square well potential (a 'quantum box') with side length L. Start from the time-independent Schrödinger equation and apply the appropriate boundary conditions." The model produced correct results, although LaTeX rendering had minor issues. You also get full access to its reasoning, so you can follow step by step how the model arrives at the solution.
Everything runs locally, so our data stays private and no cloud services are required. Text-Generation-WebUI is perfect for experimenting with AI, studying, or nerdy fun.
Pro tip: try giving the model a challenging physics, math, or coding problem and check its reasoning along with the answer.
Link:
https://github.com/oobabooga/text-generation-webui
What problem would you test with your local language model, a math derivation, a story, or code snippet?
#FOSS #OpenSource #Linux #CLI #Terminal #TextGenerationWebUI #LocalAI #LanguageModels #GPTOSS #4bitQuant #NLP #NerdContent #AdventCalendar #OpenTools #FOSSAdvent #CommandLine #CreativeAI #AI #Fediverse #TechNerds #adventkalender #adventskalender #KI #ArtificialIntelligence #KünstlicheIntelligenz #llm