3 weeks in and I can say the #homeassistant yellow works great with the CM5. All I did was pop out the CM4, move the little heatpads and flashed a new version of HAOS on to the SSD using the Pi imager.
I didn't *need* to reinstall, but my old install was severely full of cruft after years of tinkering.
The CM5 is fast. HA runs as quick as it does on a low end miniPC. Uses about 5W running off POE.