Skip to content

feat(visualization): export the charts drawn over a field or a sequence - #8504

Draft
kz930 wants to merge 2 commits into
apache:mainfrom
kz930:feat/standalone-viz-domain-plots
Draft

kz930 wants to merge 2 commits into
apache:mainfrom
kz930:feat/standalone-viz-domain-plots

Conversation

@kz930

@kz930 kz930 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Ten charts implement StandaloneCodeGenerator: Filled Area, Contour, Dumbbell, Time Series, Quiver, Choropleth Map, 3D Scatter, Volcano, Gantt and Carpet. What they share is the question they answer, which is how a value varies across a domain rather than where a row sits on an axis.

Each emits the plotly its executor emits and writes the page to the file the translator names for it, so a plan holding two charts writes two files rather than one overwriting the other. A chart that cannot be drawn writes the reason to that page rather than printing it: the engine's operator yields a page either way, so an export that printed instead would leave the run with no file where the workflow had one.

#8346 keeps the charts drawn on a coordinate system.

Any related issues, documentation, discussions?

Part of #8325, 17 of 27; that issue lists the set in order.

Two behaviour fixes travel with these charts, both on operators that moved here: the Filled Area Plot requires its line group once the switch that reads it is on, and the Contour Plot says what it cannot contour instead of ending the run. Closes #8283, closes #8076.

Closes #8503, the task this change is the whole of.

How was this PR tested?

Each chart asserts the block it emits in its own spec. Contour, Filled Area and Carpet additionally run the generated Python, since what they are worth checking is that the page comes out at all on the inputs that cannot be drawn: collinear points, a group with no line to draw, a table emptied by dropping nulls.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

🤖 Generated with Claude Code

Ten charts implement the trait: Filled Area, Contour, Dumbbell, Time
Series, Quiver, Choropleth Map, 3D Scatter, Volcano, Gantt and Carpet.
What they share is the question they answer, which is how a value varies
across a domain rather than where a row sits on an axis.

Split out of the coordinate-system change on review, which had grown past
what one reading can hold.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @eugenegujing, @aglinxinyuan
    You can notify them by mentioning @eugenegujing, @aglinxinyuan in a comment.

@codecov-commenter

codecov-commenter commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.59%. Comparing base (1fbd346) to head (50f987e).
⚠️ Report is 36 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8504      +/-   ##
============================================
- Coverage     93.69%   93.59%   -0.10%     
- Complexity     4826     4837      +11     
============================================
  Files          1209     1212       +3     
  Lines         49871    50004     +133     
  Branches       6099     6112      +13     
============================================
+ Hits          46727    46802      +75     
- Misses         1652     1699      +47     
- Partials       1492     1503      +11     
Flag Coverage Δ *Carryforward flag
access-control-service 71.78% <ø> (-8.40%) ⬇️
agent-service 99.32% <ø> (ø) Carriedforward from 539c969
amber 89.91% <ø> (-0.01%) ⬇️ Carriedforward from 539c969
computing-unit-managing-service 77.14% <ø> (ø) Carriedforward from 539c969
config-service 87.37% <ø> (+0.24%) ⬆️
file-service 83.65% <ø> (ø) Carriedforward from 539c969
frontend 96.16% <ø> (ø) Carriedforward from 539c969
notebook-migration-service 83.73% <ø> (ø)
pyamber 98.47% <ø> (ø) Carriedforward from 539c969
workflow-compiling-service 77.19% <ø> (ø) Carriedforward from 539c969

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

📊 Arrow Flight E2E bench

(no arrow-flight-e2e.csv in artifact)

Full dashboard · Workflow run

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The chart outputs use separate files, and contour errors no longer stop the workflow. Looks good.

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filled Area calculates tolerance differently in standalone mode. The native path uses len times 5 divided by 100. The standalone path divides len by 100 first. For 150 groups this gives 7 versus 5. Please use the same formula and add a boundary test.

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carpet Plot still differs on bad input. The native path returns an error page for missing columns or values that cannot be converted to numbers. Standalone calls dropna and astype first and crashes. Please keep the same error behavior and add both tests.

The exported block for the filled area plot rounded its tolerance the
other way round: five percent of the groups is (n * 5) // 100, not
(n // 100) * 5, and at 150 groups that is 7 rather than 5. The script
refused three tables the operator draws. The loop around it is now the
operator's own, down to the break that ends it.

The carpet plot dropped nulls before it looked for the columns to drop
them from, so a column that is not in the table reached pandas as a
KeyError and a value that is not a number reached astype as a
ValueError. Both ended the whole exported run, where the operator
answers each of them with a page saying what is wrong. The block now
checks in the operator's order and writes the same pages.

Both are checked by running the exported code rather than by reading
it: the filled area plot at the boundary the tolerance sets, 7 disjoint
groups of 150 drawn and 8 refused, and the carpet plot over a missing
column, a column of words, an empty table, a table of nulls, and a
table it can draw.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kz930

kz930 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Both are real and both are fixed: Filled Area now uses the operator's (len(grouped) * 5) // 100, and Carpet Plot checks the columns before it drops nulls and catches the conversion, so it writes the same pages the operator yields instead of raising. Both are covered by running the exported code, at the tolerance boundary (7 disjoint groups of 150 drawn, 8 refused) and over a missing column, a column of words, an empty table, a table of nulls and a table it can draw.

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Filled Area and Carpet changes address the earlier result and error handling differences. Looks good.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

3 participants