I was starting to think that slow internet speeds with Virgin Media might be my Intel Mac mini because others have reported similar with Macs, but I'm now hopeful that it was OneDrive gobbling all the bandwidth, and that limiting upload/download rates will fix it. 🤞 It wasn't just a bit slower, rates had dropped to an unusable fraction of a Mbps.
(125 KB/s was still causing slowdown, so I've gone done to 50 KB/s, which seems to be behaving better.)