Signal is lacking the option to block receiving voice messages.

If you want to talk to me, call me.
If you want to talk into your phone, use dictation.