[SPARK-58894][SQL] Detect cyclic view references in nested subqueries

[manuzhang]

What changes were proposed in this pull request?

Update ViewHelper.checkCyclicViewReference to traverse complete expression trees when
looking for subquery expressions. Add a regression case where a scalar subquery that
references the altered view is nested below an equality predicate.

Why are the changes needed?

The current check only inspects top-level plan expressions. It detects an EXISTS
subquery when the subquery is the expression root, but misses scalar and other subqueries
nested inside larger expressions. A recursive view definition can therefore be accepted
and fail later during view resolution with the maximum nested-view-depth error.

Does this PR introduce any user-facing change?

Yes. CREATE OR REPLACE VIEW and ALTER VIEW now reject recursive references from
subqueries nested inside larger expressions with RECURSIVE_VIEW, matching root-level
subquery behavior.

How was this patch tested?

Added a regression assertion to SimpleSQLViewSuite and ran:

SPARK_LOCAL_IP=127.0.0.1 build/sbt \
  'sql/testOnly org.apache.spark.sql.execution.SimpleSQLViewSuite -- -z "correctly handle a cyclic view reference"'
build/sbt 'sql/scalastyle' 'sql/Test/scalastyle'

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Codex (GPT-5)

[manuzhang]

@szehon-ho @aokolnychyi This is why we can't use Spark's ViewHelper.checkCyclicViewReference in apache/iceberg#14984. Please help review. Thanks!

[zahed1994]

Nice catch and elegant fix @manuzhang!

Review Summary:

• Root Cause: plan.expressions.foreach { expr => expr match ... } previously only checked top-level expressions where the expression root itself was a SubqueryExpression. Nested subqueries inside binary/complex expressions (e.g. WHERE id = (SELECT ...) where root is EqualTo) bypassed cycle detection.
• Fix: Changing to expr.foreach properly traverses the entire expression AST down to any nested SubqueryExpression nodes.
• Test Case: Verified the new ALTER VIEW view1 AS SELECT * FROM jt WHERE id = (SELECT id FROM view2) regression test in SQLViewSuite.

LGTM +1!

[uros-b]

Thank you @manuzhang and @zahed1994!

[szehon-ho]

Nice find. I think my only concern, this is a behave change. The old view probably will not run. But this prevent bad view from being created and it could today (and later fixed before running). but leave up to @uros-b @cloud-fan @dongjoon-hyun and others

[cloud-fan]

0 blocking, 0 non-blocking, 0 nits.
The patch is narrowly scoped, and its traversal semantics, shared entry paths, error behavior, and regression coverage are consistent.

Verification

Source-level verification traced Expression.foreach to its recursive child walk, confirmed that SubqueryExpression.plan is reached only by the explicit validator recursion, and checked the v1 and v2 callers of the shared helper. The regression assertion exercises the previously missed equality-wrapped scalar subquery and verifies RECURSIVE_VIEW. I did not rerun the test suite locally.

[cloud-fan]

Merge Summary:

• merged into master 02c21a6
• merged into branch-4.x 4528b8f
• merged into branch-4.3 ef4fd13

Posted by merge_spark_pr.py