denhamFeb 22

As I experiment with running local #llm on my #framework desktop, having 128GB of ram certainly gives you lots of options. I can run some large models, but they're generally quite slow.

#lmstudio #ollama #frameworkdesktop #qwen #qwen35

Show threaddenhamFeb 22

My current set-up is to run two 'smaller' models simultaneously, a planning and coding model.

Qwen3-32B is my 'planner' model, which has good reasoning/instruction following capabilities.

Qwen3-Coder-30B-A3B is my 'coding' model, for coding, tool calling and debugging.

Show threaddenham

I'm running #opencode in the terminal, which by default has two primary agents, plan and build agent. This setup pairs nicely with that.

#ohmyopencode

Feb 22 at 7:30pmWeb
Show threadPatrick McDavidFeb 22
@denham can you quantify the slowness? Is it essentially slow as perceived to a high end commercial coding AI?
Show threaddenhamFeb 23
@ehippy It's slow on the order of minutes for a response, won't be as accurate and hallucinating is still an issue - which all impacts speed of resolving tasks. I won't be cancelling my #claude subscription just yet, but there are a subset of use cases it's ideal for.
Show threadPatrick McDavidFeb 23
@denham are you doing anything with multiple models policing each other?
Show threaddenhamFeb 24

@ehippy can't say that I am specifically. ohmyopencode supports configuring models by task types, so a local model for tasks categorized as 'quick' would be fine, while you'd have an Opus model orchestrating things.

Are you aware of anything like you're suggesting?

Show threadPatrick McDavidFeb 24
@denham I guess not specifically. I’ve got one of these framework desktops and I want to use it for the same coding use case I think. Can we not set up oh my open code to route to a checker agent at the end or a security agent and lint/type check whatever kinda expert at the end. You’re ahead of me at this point. 🤣