A bridge over a river never crossed

When I first started my forever project, a peer to peer file sync software [https://github.com/ferd/ReVault/] using Interval Tree Clocks [https://ferd.ca/interval-tree-clocks.html], I wanted to build it right. That meant property-based testing everything, specifying the protocol fully, dealing with error conditions, and so on. Hell, I grabbed a copy of a TLA+ book [https://link.springer.com/book/10.1007/978-1-4842-3829-5] to do it. I started a document where I noted decisions and managed to write up a pretty nifty file-scanning library that could pick up and model file system changes over trees of files. The property tests are good enough to find out when things break due to Unicode oddities, and I felt pretty confident. Then I got to writing the protocol for a peer-to-peer sync, with the matching state machines, and I got stuck. I couldn't come up with properties, and I had no idea what sort of spec I would even need. Only one sort of comparison kept popping in my mind: how do you invent the first telephone? It’s already challenging to invent any telephone (I would assume so at least), but you’d have the benefit of having existing infrastructure and networks, on top of having other existing telephones to test it with. But for the first telephone ever, you couldn’t really start with a device that has both the mouthpiece and the ear piece in place, and then go “okay now to make the second one” and have a conversation once they're both done. In some ways you have to imagine starting with two half-telephones, but your both sides have a distinct half. You start with a part to speak in and a part that goes on the other side and sort of gradually build up a whole pair I guess? An actor portraying Alexander Graham Bell speaking into a early model of the telephone for a 1926 promotional film by AT&T, public domain. The phone is a simple conical part in which the actor is speaking, attached to a piece of wood, with no ear piece at all [https://i.imgur.com/YiHGmTw.png] This was the sort of situation I was finding myself in for the protocol: I wanted to build everything correctly the first time around, but I had no damn idea about how to wire up only one fine half to nothing just to figure out what shape exactly should a whole exchange have. I had written protocols before, I had written production-grade distributed software before, there was prior art for this sort of thing, but I had never built this specific one. To put it another way, I wanted to build a bridge that would be worth it, but I was trying to do it over a river I had never once crossed. I could imagine the finished product’s general shape and purpose, and had even built bridges before, but not over this specific river. Hell, without having gone over the gap once end-to-end, I had no idea what the other side looked like. As it turns out, forcing myself to prototype things and make a very bad version of the software was the most effective way to make a slightly less bad version of it that follows. And then a slightly better one, and another. This was iterative development winning over careful planning. I’m nearing the point where I have a shoddy wooden bridge that I can cross over on. It’s real crappy software, it doesn’t deal with errors well (it’s safe and doesn’t break things, but it’s also unreliable and crashes or hangs all the time), it’s not very fast, and it's downright unusable. But then I also figure that either way I’ll have a lot more infrastructure to work with. And once I’m through with the mess, I can maybe design a nicer form of it. Building the bridge as you cross the river for the first time is a paralyzing thought, and despite all my wishes about it being done right on that initial attempt, it turns out it's a pretty good idea to make that first crossing as cheap and easy to tear down—and replace—as possible. Saying "build a prototype and be ready to replace it" is a long known piece of conventional wisdom. The challenge is how crappy or solid should your prototype be? What is it that you're trying to figure out, and are you taking the adequate means for it? There is a difference between a rough sketch with the right proportions and exploring from an entirely wrong perspective. Experience sort of lets you orient yourself early, and also lets you know which kind you have in your hands. I guess I'll find out soon if all the fucking around was done in the proper direction. Funnily enough, traditional arch bridges were built by first having a wood framing on which to lay all the stones in a solid arch [https://www.youtube.com/watch?v=nJgD6gyi0Wk]. That wood framing is called falsework [https://en.wikipedia.org/wiki/Falsework], and is necessary until the arch is complete and can stand on its own. Without falsework, no such bridge would be left standing. That temporary structure, even if no trace is left of it at the end, is nevertheless critical to getting a functional bridge. Falsework centering in the center arch of Monroe Street Bridge, Spokane, Washington, 1911. An elaborate wooden structure is supporting concrete until it can self-stand. [https://i.imgur.com/lL71ihl.png] I always considered prototypes to be a necessary exploratory steps, where you make mistakes, find key risks about your project, and de-risk a more final clean version. They were dirty drafts, meant to be thrown out until a clean plan could be drawn. I thought, if I had enough experience and knowledge, I could have that clean plan and just do it right. Maybe I just needed to get over myself and consider my prototype to in fact be Falsework: essential, unavoidable, fundamentally necessary, even if only temporary. note: this text itself might just be a draft (or falsework?) for a follow-up, less rambling post on my blog.