The Internet says the best way is to encode two videos, one video holding YUV and one video holding the alpha channel, then during decoding time recombine. Not sure how to interleave them in .
(Or I could just key out one colour, but visionOS's e.g. app icon grid probably won't look good without full alpha...)

@zhuowei That's exactly how "VP6 with alpha" in Flash works. The color is premultiplied, and alpha is a separate greyscale stream. There's a constraint there that any frame in the color stream that is a keyframe (I frame) must also be a keyframe in the alpha stream. Other than that VP8/VP9 in WebM can handle alpha natively it seems...?