This message correspond to something like “bad endpoint” or bad signature version support.
like seen here frankfurt is the only one that not support signature version 2. And it’s the one I picked.
Of course after all my reserch can’t say what is signature version, it’s not obvious in the documentation. But the V2 seems to work with s3a.
The endpoint seen in the S3 interface is not the real endpoint it’s just the web endpoint.
you have to use one of theses endpoint like that
sc.hadoopConfiguration.set("fs.s3a.endpoint", "s3.eu-west-1.amazonaws.com")
But it’s work by default with US endpoint