From a lecturer viewpoint I am astonished, that this is an issue and clearly shows lack of engagement on the lecturer side.
What I'm doing in these situation is being clear, that everyone is there on a voluntary basis (as all are grown ups), and thus do not have to be there. If they rather talk, they are free to leave and not steal time and ressources off people, who want to learn. If nothing changes, I'm actually throwing people out.