Globally you can set
[…] Gradle user home directory (defined by the “GRADLE_USER_HOME” environment variable, which […] defaults to USER_HOME/.gradle) […]
Otherwise for every build manually:
-g, –gradle-user-home
Specifies the Gradle user home directory. The default is the .gradle directory in the user’s home directory.
See https://gradle.org/docs/current/userguide/gradle_command_line.html.