@W-23900223 : Hyper Query Plan: Add a plan-insights panel and improve query plan readability - #172
Open
rbrahmachary wants to merge 5 commits into
Open
Conversation
1. Report actual rows again: read statistics.output-rows / estimated-rows (with analyze.tuple-count / cardinality fallback). Actuals were silently dropped after the rename, leaving edges estimate-only.
2. Correct edge label order to estimate/actual across generic and scan edges.
3. Read runtime fields (running, cpu-cycles, ...) from statistics first, analyze as fallback.
4. Add an absolute floor to the cardinality-mismatch check so a 36-vs-0 miss no longer highlights like a 540M-vs-0 one.
5. Stop leaking the raw statistics block as a stringified-JSON property.
Enhancements:
1. New PlanInsights overlay: issue summary, category legend + counts, top scans by rows processed, and issue navigation / focus mode. The panel can be toggled (minimized to a compact header bar) to get out of the way.
2. Highlight categories with node/edge color + hover reasons: costly scan, cardinality misestimate, runtime CPU hotspot, index recommendation, index used. The index recommendation legend calls out to verify the query traffic pattern before implementing. A node that is both a costly scan and an index-recommendation candidate keeps its category fill and additionally gets the amber index-rec border, so both signals show at once.
3. Live-editable thresholds: a "How highlighting works" footer tunes every threshold and re-highlights the graph without reloading (threshold logic centralized in highlight-rules.ts; recomputed at render time).
4. Custom edges (QueryEdge) with "why highlighted" tooltips.
5. Costly scan highlighter - A scan billed far more rows than matched its restrictions — whole row groups were read to return few rows (low selectivity). The red deepens with the scan's share of all rows the plan read, so the heaviest scans stand out.
All changes are scoped to Hyper plans; all other plan types remain unchanged. The Postgres, Tableau, generic JSON, and XML loaders only stamp a new 'planSource' field — their tree structure, node/edge rendering, and behavior are otherwise identical. The PlanInsights overlay is gated on 'planSource === hyper', so non-Hyper plans render exactly as before (no summary header, no side panel).
operator produces and consumes, and by hardening a few metadata paths
that could leak placeholder text.
Column flow through set operations:
- Propagate a set operation's output-column names (its , resolved to
display names) down onto the positionally-aligned input columns, so a
union-all's inputs (maps, scans) read with the same column names as the
union itself and the flow is verifiable at a glance.
- Overwrite each set-op input's derived with the set op's
own columns (union-compatible inputs share its schema), so input counts
and names line up exactly with the union-all instead of disagreeing.
- Name a map's computed rows by their recovered output name so
the LHS labels match the node's .
Operator display:
- Relabel a bare LIMIT (empty sort criterion + cputime unlimited
filesize unlimited
datasize unlimited
stacksize 7MB
coredumpsize 0kB
addressspace unlimited
memorylocked unlimited
maxproc 10666
descriptors 1048575) from sort to
limit and give it a dedicated glyph (kept rows / cut line / dropped
rows), so it no longer reads or renders as an ordering operation.
- Order per operator: front it for set operations and
generic operators (columns-first reads best), but leave it trailing when
a block already establishes a semantic lead (join condition, group-by
keys, sort keys, map computes) so the lead isn't buried.
Robustness (avoid literal undefined from tryToString on missing fields):
- Drop aggregates with a missing instead of rendering
.
- Only set a udtablefunction's / when present.
- Only set a generic scan's when is a real string, so no
spurious undefined bucket appears in the plan-insights breakdown.
…on the
recent FORMAT JSON rework:
1. Alias flood-fill. A query's SELECT Name__c AS 'Account Name' records the
alias only at the top-level projection's output-names. Propagate that
user-facing name across 'same logical column' links (set-op values, map
passthroughs, scan renames, group-by keys) via an undirected fixpoint so
every operator carrying the column reads the SQL alias, not the base name.
Scans keep the base name and annotate it (Name__c -> Account Name).
2. Carry real column names onto GroupByKeyN. A group-by emits grouping keys
as fresh opaque IUs that otherwise render as a group-key placeholder
downstream. Link each plain-column key to its source column so the real
name flows onto the produced key (join conditions, explicit-scan re-reads).
Computed keys have no single source column and are left unlinked.
3. Order output columns by direct parent usage. Compute the IUs an operator
reads in its OWN expressions (join condition, key-expressions, map values,
sort keys) -- stopping at nested-operator and passthrough-projection
boundaries -- and lead each node's column preview with the ones its parent
consumes, so the join/grouping key is visible without scrolling a wide row.
Introduce an interactive insights layer for Hyper plans:
- PlanInsights panel: legend, ranked 'top offenders' lists (costly scans,
CPU/memory hotspots), query-error banner, search callout, and adjustable
highlight thresholds with focus-issues navigation.
- highlight-rules: shared threshold model + heat-shade/reason helpers driving
costly-scan, high-volume-scan, cardinality, and runtime/memory hotspot tints.
- hyper.ts: humanize expressions and resolve IU references to real column
names/aliases, so predicates read like the SQL (e.g. count2 > 1).
- Render edge row-count labels via a portaled QueryEdge (replacing ColoredEdge)
as opaque pills; widen inter-layer spacing for legibility.
- Flag runtime-error nodes with a red border and surface them in focus mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes Hyper query plans substantially easier to read and reason about, adding an interactive insights layer on top of the graph plus a number of rendering and naming improvements. Scoped to the Hyper loader and the shared UI; other loaders (json/postgres/tableau/xml).
Plan Insights panel
Highlighting
Readability of expressions & columns (hyper.ts)
Edges & layout
Screenshot
Recording
https://drive.google.com/file/d/1xV9CSk_Y1ajmCi8KIdyPDmtrZ_gATi-P/view