Skip to content

Share the native-reader admission checks between the core Parquet scan and the Delta contrib #5944

Description

@dwsmith1983

What is the problem the feature request solves?

DeltaScanSupport in contrib/delta-spark repeats three admission checks that core's native Parquet scan already performs: the input-file-expression check, the vectorized-reader-setting check, and the nested-default check. A future fix to any of them needs a separate Delta update.

Describe the potential solution

A shared native-reader admission helper in core that both scans call, leaving Delta's protocol checks, deletion-vector row-index exception, liveness checks and stricter encryption fallback explicit in the contrib. Filesystem policy stays with #5658. A parity test covering the common rejection cases for ordinary Parquet and Delta, plus supported-scan controls, protects the boundary.

Additional context

Raised in the #5365 review (item 2); filed as a follow-up so the refactor of the shared core path gets its own review.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions