Mastodawn

Federico Viticci

Not sure who's going to find this useful given the model's...shall we say, "constraints"… but here's a CLI to use Apple Intelligence's Foundation on-device model from the shell:

https://apfel.franzai.com/

apfel - Your Mac Already Has AI

Your Mac already has AI. Apple ships a language model with macOS -apfel unlocks it with one brew install. No downloads, no API keys, no config. The fastest path to local AI.

apfel

Paul_a_phillips 1d ago

@viticci This is actually pretty amazing.

1. brew install Arthur-Ficial/tap/apfel
2. brew install --cask macai
3. apfel --serve
4. Run the macai app and configure as shown in the screenshot (URL is http://127.0.0.1:11434/v1/chat/completions)

Boom, you have a fully local, offline, private, and environmental-friendly chatbot app for basic tasks at zero cost.

@lonzo @viticci What are you using to connect to Apfel there?

@ischack @viticci https://macai.chat/

macai - Your AI Assistant for macOS

A powerful, native AI assistant for macOS. Chat with efficient models directly from your desktop.

@lonzo @viticci Wow, Must have been really tired yesterday already. Sorry for the stupid question. 🙃

@ischack macai, it's the second item you install with brew

Rafi Saar 1d ago

@lonzo @viticci Thanks!
May I suggest to run step 3 in the background so the terminal isn’t locked:
3. apfel —serve &
And maybe also make sure it runs again at every reboot of the Mac?

@rsaar @viticci Yeah makes sense, I just wanted to outline the steps to get started ✌️

tim.fidd 1d ago

@lonzo @viticci You can sort of use this in openclaw. But as context window of the AFM in apfel is limited to 4096k, there maybe not much use of this.

@timfidd @viticci Quoting their FAQ: ”apfel originally started as an attempt to run OpenClaw on Apple Intelligence in ultra token-saving mode. That did not work out because of the 4K context window.“ 😉

tim.fidd 1d ago

@lonzo @viticci Yes I know, but it’s still possible if you spawn apfel as an agent in openclaw to run small tasks.

tim.fidd 1d ago

@lonzo @viticci So in theory you could add a heartbeat or cron job to run say mac-narrator via apfel as a spawn agent in openclaw in a heartbeat or cron job. That would work.

tim.fidd 1d ago

@lonzo @viticci Running it via a spawned agent as a heartbeat or cron job overcomes openclaw giving it its full context, so openclaw only requests the context it asks for when it spawns instead of its full memory etc. so as long as that is less than 4096k it works fine.

tim.fidd 1d ago

@lonzo @viticci So, I could ask openclaw, “use apple-fm, what is the capital of Lithuania” in this case apple-fm is my alias to the apfel model. It will use apfel and the apple foundation model locally and send me the response.

tim.fidd 1d ago

@lonzo @viticci Maybe mac-narrator at some time interval to send a message to Telegram?

tim.fidd 1d ago

@lonzo @viticci I.e. as a spawned agent in a heartbeat or cron job.