feat(storage/opendal): route oss:// URLs through the S3 backend

[plusplusjiajia]

Which issue does this PR close?

N/A

What changes are included in this PR?

Aliyun OSS exposes an S3-compatible API, and the other two Iceberg implementations already read oss:// tables through their S3 code path with canonical s3.* properties:

• pyiceberg: constructs a pyarrow.fs.S3FileSystem for oss:// locations.
• Java S3FileIO: accepts any URI scheme by design — the scheme is a label, transport is always the AWS S3 SDK.

iceberg-rust routes oss:// to opendal's native Oss service instead, which reads a separate oss.* namespace and can't be driven by the same credentials. Align with the rest of the ecosystem.

[CTTY]

Hi @plusplusjiajia thanks for the contribution! Logic wise this makes sense, but I'm concerned that this will confuse users that OSS storage has two modes. Do you think we should just keep one?

[CTTY]

Suggested change

	#[cfg(feature = "opendal-s3")]
	#[cfg(all(feature = "opendal-oss", feature = "opendal-s3"))]

Only having s3 feature flag will introduce oss related code to non-oss users

[plusplusjiajia]

@CTTY Fixed.

[CTTY]

I'm not very familiar with OSS storage so not sure about it: Is the fallback actually necessary? I'm thinking if using OSS exactly like S3 is the common case, maybe we can just use OpenDalS3Storage directly?

[plusplusjiajia]

@CTTY Agreed, removed it. OSS is S3-compatible and the Java/Python Iceberg implementations both route OSS through S3FileIO without a native fallback.

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

[Xuanwo]

iceberg-rust routes oss:// to opendal's native Oss service instead, which reads a separate oss.* namespace and can't be driven by the same credentials. Align with the rest of the ecosystem.

Instead of trying to route OSS to S3, I suggest allowing OSS to parse options from the S3 config.

OpenDAL's OSS support is natively implemented, which means it works with Aliyun OSS's RAM out of the box. Switching to S3 services prevents users from working with Aliyun's OIDC or assume roles.

[plusplusjiajia]

Hi @plusplusjiajia thanks for the contribution! Logic wise this makes sense, but I'm concerned that this will confuse users that OSS storage has two modes. Do you think we should just keep one?

Thanks for the feedback! I initially removed it, but @Xuanwo raised a valid concern — routing everything through S3 would prevent users from using Aliyun's native RAM/OIDC/assume-role auth. So I restored the native OSS path as a fallback: when s3.* config is present (e.g. REST catalog vended credentials), we route through S3; otherwise we use the native OSS backend. This keeps both use cases working.

[plusplusjiajia]

iceberg-rust routes oss:// to opendal's native Oss service instead, which reads a separate oss.* namespace and can't be driven by the same credentials. Align with the rest of the ecosystem.

Instead of trying to route OSS to S3, I suggest allowing OSS to parse options from the S3 config.

OpenDAL's OSS support is natively implemented, which means it works with Aliyun OSS's RAM out of the box. Switching to S3 services prevents users from working with Aliyun's OIDC or assume roles.

Good point! I've added a dual-path approach: has_s3_config() checks for s3.endpoint or s3.access-key-id — if present, routes through S3; otherwise falls back to the native OSS backend so RAM/OIDC/assume-role still works.

[plusplusjiajia]

Hi @CTTY @Xuanwo, mind taking another look when you get a chance?

The dual-path resolver should cover both your points: has_s3_config() routes through S3 only when s3.* vended creds are present, else falls back to native OSS (keeps RAM/OIDC working — thanks @Xuanwo). Routing is gated behind all(feature = "opendal-oss", feature = "opendal-s3"), so S3-only builds pull in no OSS code (@CTTY). Thanks!

[plusplusjiajia]

@CTTY @Xuanwo friendly ping — the dual-path resolver should address both of your points:

• has_s3_config() routes through S3 only when s3.* vended creds are present, else falls back to the native OSS backend — so Aliyun RAM/OIDC/assume-role still works (@Xuanwo).
• Gated behind all(feature = "opendal-oss", feature = "opendal-s3"), so S3-only builds pull in no OSS code (@CTTY).

Mind taking another look if it looks good? Thanks!

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

[plusplusjiajia]

@Xuanwo — on not breaking native Aliyun auth: the resolver is dual-path. oss:// only routes through S3 when s3.* config is present (REST vended credentials); otherwise it falls back to the native OSS backend, so RAM/OIDC/assume-role keep working. Gated behind all(feature = "opendal-oss", feature = "opendal-s3").

@CTTY — would appreciate another look

[plusplusjiajia]

@CTTY @Xuanwo Sorry to ping again — I'd rather check the reasoning than push another revision.
The dual path tries to satisfy both points at once: oss:// goes through S3 only when s3.* config is present, so native RAM/OIDC still works when it isn't (@Xuanwo), and it's feature-gated so S3-only builds pull in no OSS code (@CTTY). The cost I've found since: one props map serves every scheme, so a config carrying both s3.* and oss.* is ambiguous — pyiceberg and Java avoid that by routing oss:// through S3 unconditionally.
Does that tradeoff still look right, or would you rather collapse it to a single path? Happy to implement either.