p2panda is designed with a clear separation of the "event delivery" and "event processing" layers. p2panda applications should not worry about _where_ their messages come from but only how they are processed.

We make sure that other nodes who are interested in the same topic are confidentially discovered, to sync up on all missed messages, to deliver them to the processing layer.

This gives them flexibility to run on top of any infrastructure. We also call this a "Walkaway Stack".

So far we do this over the Internet using iroh to manage direct p2p connections, but we're prepared to offer the same level of abstraction for running over Radio-Protocols like LoRa or BLE, including mesh topologies.

Application developer will mostly use the API to monitor and inspect networking and sync activity, restrict access and select available transports (QUIC, Tor, Mesh Networks, BLE, ..) while we make sure data is delivered as securely and privately as possible.

Any message you send is delivered on top of our "Operation" data-type. That's an single-writer append-only log with some cool features: You can have as many logs as you want, you can remove any past entries in the log (pruning) and it's robust to forks. It's a simple CRDT actually if you look closer!

With append-only logs we are prepared to offer the same functionality over radio-based meshes as sync is cheap (state vector is exchanged in one message, constant-time) and broadcast-only friendly.

We can compose additional "system-level" features on top of Operations, by extending the header with information to help garbage collection / pruning, tombstones to delete multiple logs, vector clocks for causal ordering of messages for multi-writer documents, data to aid key agreement for group encryption and more!

It would also be possible to add existing solutions, like Willow's Meadowcap or UCAN, to check against permissions with certificate capabilities.

Developers generate a Topic (random bytes) to publish messages into it and subscribe to for receiving everyone else's messages. This is PubSub!

Topics are a way to group your data, a topic is for example a Reflection text-document, or a single chess game session, or a chat room.

Later we also want to divide the space up even more into human-readable strings you can then filter with wildcards (*).

This is partial replication: You only sync the data you need.

Now we have a system where we publish, sync and persist operations grouped by topics on top of any p2p network.

With this data being available to us now we want to make sure to correctly process and stream it towards the application layer. For this we're borrowing many conceps from general "event streaming" theory, you might know this from NATS JetStream, Kafka or something like that, with the difference that we're not aiming for "big data centres" but decentralised and autonomous p2p networks.

When being subscribed to a topic you have something like a stateful "stream consumer" now.

Every operation which runs through that stream gets processed, first on an internal system-layer where we check the log-integrity, prune the log on demand, order operations causally and later we'll also decrypt them here.

After this we're forwarding the message to your application-layer, with a bunch of meta data and debugging info attached.

The application-layer can continue processing the message delivered by this operation, now based on your own application logic. Often it'll be something like "move the pawn to field E4", "change the title of the document" or a chat message, or a Text CRDT etc.

Finally we want to acknowledge the operation. This will mark the operation internally as "acked" and next time we subscribe to the same stream we will not receive the message anymore. It's something like a "read receipt".

By default we're automatically acking messages from the stream for you, since this is a more advanced feature, but why is this important?

Imagine your application crashes mid-way or you use it on a mobile phone where it got closed while it was processing something. If we didn't have a chance to ack the message it'll be re-played next time when the app opens again, giving us a chance to make sure we've processed every message at least once.

Re-plays are also cool for rolling out updates or fixes btw! We can re-play everything or move the cursor back to a point where we need to re-materialise our application's state because something went wrong in the past.

Okay, enough about event streaming, you get the idea now that p2panda's high-level API is actually something like a feature-complete event streaming system, but with an eventual consistency guarantee and causal ordering.