Public service announcement from professional cyclist Tibor Del Grosso: just ask before you start following someone out on the road. It's the polite thing to do. Be safe, be mindful, don't be a creep.
The pros attract this kind of behaviour but the advice works exactly the same for any person or group you encounter on the road: want to ride along? Just ask.
From his training camp vlog at https://youtu.be/8hkmHoM0iek