strickvlI keep thinking about staff engineers managing ML platforms who barely have time for the work they already have. Would something like this actually save you time? And is email the right place for it, or would you rather get this in Slack or somewhere else?
Matches one of the ways I like to learn - not just reading docs, but actually debugging and fixing things myself. Different people learn differently. For me, poking at broken code from multiple angles makes concepts stick. (And reading the docs, and writing or blogging about things, etc etc!)
Nothing fancy - just an experiment in interactive learning.
https://github.com/strickvl/zenlings for the repo. YMMV!
Each traverse_node is a separate step, created at runtime, with its own artifacts, retries, and lineage.
Next week: how ZenML approaches dynamic pipelines specifically.
These aren't new problems. We solved (or at the very least surfaced and discovered) them in ML pipelines years ago. But as people productionise agents, all this infrastructure becomes critical again.
So my question isn't "are dynamic pipelines good or bad?" It's: when do you actually need them, and when are you just avoiding the work of understanding your domain?
More tomorrow on where dynamic genuinely makes sense.
What's next: I'm thinking about MCP apps for interacting with ZenML deployments, custom platform-engineer dashboards (failure rates by team, run overviews across projects), and more.
What would you want an MCP app for? Drop ideas here, open a GitHub issue, or find us on Slack. It's open source; PRs welcome too!
The concept is v exciting — dashboards and interactive views instead of text walls. Also you presumably don't pollute your context window when you're using the interactive app. But I'm not yet sure the chat window is always the right home for rich UIs, and I'm assuming it won't be the final form these apps take.
We'll keep building and share what we find. Demo video and some screenshots tomorrow.
I hadn't thought much about this until someone asked whether ZenML locks you into Python-only codebases. The answer is more nuanced than I expected.
Put together an example repo with a multi-stage Dockerfile for remote orchestration and some sample earnings transcripts. Still want to try adding Rayon for parallel processing within a single step.
Example repo: https://github.com/zenml-io/rust-pipeline/
https://github.com/zenml-io/mcp-zenml for the source code and instructions but you can also get instructions on how to install it with pre-populated auth tokens etc from within your ZenML Pro web UI.