@wimpy I see it as an issue on both sides. A #cdn should handle huge spikes in traffic. I would look at more details of how the CDN is being leveraged. There are some settings to tweak to improve performance.
On the other side, #Mastodon should fix this issue. It's sounds like a bad design smell. Each instance should cache the link preview at the minimum. It shouldn't hit the source for per follower. #twocents