docs: remove references to closed and fixed issues from compatibility guide#4856
Merged
Merged
Conversation
… guide The compatibility guide linked several closed issues and documented a few behaviors that have since been fixed. Remove the stale entries and drop links to closed issues while keeping the descriptions of genuine current limitations. - Remove the TimestampLTZ to TimestampNTZ scan bullet (apache#4219) and the duplicate-field-names bullet; both described Comet matching Spark rather than a divergence. - Remove the Hour/Minute/Second TimestampNTZ bullet (apache#3180), fixed by apache#4753. - Drop closed-issue links (apache#286, apache#313, apache#1947, apache#4051, apache#2649, apache#4352, apache#4729) from entries whose surrounding text still describes real behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A
Rationale for this change
The Parquet/expression/operator compatibility guides linked a number of closed issues and, in a couple of places, documented behaviors that have since been fixed. Pointing readers at closed issues (including closed EPICs described as "we are tracking the work") and listing already-fixed behavior as a limitation is misleading. Compatibility docs should describe current divergences from Spark and link only to open tracking issues.
What changes are included in this PR?
scans.md: TimestampLTZ read as TimestampNTZ (closed native_datafusion more permissive than Spark 3.x when reading Parquet TimestampNTZ columns #4219) and duplicate field names in case-insensitive mode.datetime.md: Hour/Minute/Second applying timezone conversion to TimestampNTZ inputs (closed hour/minute/second expressions incorrectly apply timezone conversion to TimestampNTZ inputs #3180, fixed in feat: support TimestampNTZ inputs natively for hour/minute/second #4753).cast.md: closed EPICs [EPIC] Fully support ANSI mode #313 and [EPIC] Spark-compatible cast / try_cast operations #286.spark-versions.md: collation support (Fix listagg-collation.sql test in Spark 4.0.0 #1947, Broadcast hash join and sort-merge join can produce incorrect results on non-default collated keys [Spark 4] #4051).datetime.md:TruncTimestamp(date_trunc incorrect results in non-UTC timezone #2649).scans.md: disabled vectorized reader (native_datafusion: tests asserting parquet-mr's permissive overflow/narrowing behavior cannot be made to pass #4352).operators.md: sliding-frame decimalsum/avg([Bug] SUM(decimal) over a sliding window frame returns wrapped out-of-range value on overflow instead of NULL #4729).After this change, the compatibility guide links only open issues.
How are these changes tested?
Documentation-only change. Verified that no links to the removed closed issues remain and that every issue still linked from the compatibility guide is open.