New blog post! We just released a new Rust crate for cross-platform screen capture called CrabGrab, and the first thing we built was an AI assistant that can "see" your screen. Using Tauri, GPT-4, and CrabGrab we built Snippy!

https://www.timdbg.com/posts/snippy-assistant-with-eyes/

Snippy: An AI Assistant With Eyes

While I no longer work on WinDbg, I still spend a lot of time thinking about how to make tools so people can build things faster. With WinDbg, I tried to do that by putting more debugging power at people’s fingertips in a way that was easier to use. Recently, everyone is looking for ways to use LLMs to build things faster. But for most people using something like ChatGPT means pasting text back and forth between your different tools.

TimDbg

You can use the CrabGrab crate with just a few lines of code to capture frames from a whole screen or individual applications. And it's cross-platform on macOS and Windows! There are a ton of gotchas with screen capture, and CrabGrab makes it easier.

https://crates.io/crates/crabgrab

crates.io: Rust Package Registry

The Snippy assistant is just a demo of how to use CrabGrab, and I'd love to see other folks make a more fully feature assistant out of it. Contributions welcome on both Snippy and CrabGrab. Both repos are on GitHub.

https://github.com/AugmendTech/Snippy

https://github.com/AugmendTech/CrabGrab

GitHub - AugmendTech/Snippy: CrabGrab+Tauri Example App

CrabGrab+Tauri Example App. Contribute to AugmendTech/Snippy development by creating an account on GitHub.

GitHub