If I was to do this though I wonder how I'd handle physics.

Two methods that come to mind, first one is opt-in like how Aeronautics does it. Second one is opt-out using like an anchor crystal.

Meaning blocks not connected to the world (within a certain mass, I'm not that good of a dev) would have physics. But then using like anchor crystals you could lock blocks in place. Allowing for magical sky islands to still naturally spawn or for you to anchor your own build.