What is the problem the feature request solves?
The SPI returns CometS3Credentials with non-null access/secret keys, so it cannot express
"anonymous". When the built-in Hadoop adapter is enabled globally and a bucket's chain resolves to
AnonymousAWSCredentialsProvider (common for public datasets), the adapter fails with a clear error
and users must opt the bucket out with an empty fs.s3a.bucket..comet.credential.provider.class.
Supporting anonymous through the SPI (a sentinel credential or an isAnonymous flag the native reader
maps to skip-signature) would let the adapter match Spark for these buckets. See #6023.
Describe the potential solution
No response
Additional context
No response
What is the problem the feature request solves?
The SPI returns CometS3Credentials with non-null access/secret keys, so it cannot express
"anonymous". When the built-in Hadoop adapter is enabled globally and a bucket's chain resolves to
AnonymousAWSCredentialsProvider (common for public datasets), the adapter fails with a clear error
and users must opt the bucket out with an empty fs.s3a.bucket..comet.credential.provider.class.
Supporting anonymous through the SPI (a sentinel credential or an isAnonymous flag the native reader
maps to skip-signature) would let the adapter match Spark for these buckets. See #6023.
Describe the potential solution
No response
Additional context
No response