Version 0.34.0 of the gradle-maven-publish-plugin finally comes with configuration cache support. Dropping support for Sonatype OSSRH unblocked us from waiting for Gradle to resolve the underlying issue on their side.
https://github.com/vanniktech/gradle-maven-publish-plugin/releases/tag/0.34.0