Cost and Factorization Analysis Utility#577
Open
ajay-mk wants to merge 10 commits into
Open
Conversation
A standalone utility that estimates the cost of evaluating a serialized tensor equation. Reads a JSON driver (space sizes + equation files), strips the (anti)symmetrizer, flattens, then per summand optimizes and binarizes once, reusing that tree for the intermediate catalog, cache simulation, and --dump_tree, mirroring MPQC's SeQuantEngine. Emits a Markdown report on largest intermediates, most expensive contractions, shape census, peak storage, FLOPs, and a gated cache simulation. Includes a README and CCSD/DF example drivers.
Diff generated reports against frozen *.md.expected via run_tests.cmake (volatile lines stripped): a spin-orbital CCSD R2 (a Sum of terms, with a --dump_tree check) and a single density-fitted product (the non-Sum path).
Krzmbrzl
reviewed
Jul 9, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new standalone cost_analysis utility under utilities/ that reads a JSON driver, optimizes/binarizes SeQuant equations, and emits a Markdown report (plus optional tree dumps) along with reference-style CTest coverage.
Changes:
- Added the
cost_analysisexecutable with JSON-driven configuration, equation parsing, optimization/binarization, cataloging, and optional cache simulation. - Added Markdown report generation and accompanying documentation/examples with frozen
*.md.expectedfixtures. - Integrated the tool into the build and added CTest-based golden-file regression tests.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| utilities/cost_analysis/run_tests.cmake | CMake script to run the utility and compare generated reports to expected fixtures (with volatile lines stripped). |
| utilities/cost_analysis/report.hpp | Declares report-writing and tree-string serialization helpers. |
| utilities/cost_analysis/report.cpp | Implements Markdown report formatting, table generation, and tree stringification. |
| utilities/cost_analysis/README.md | Documents the driver schema, workflow, report contents, and tests. |
| utilities/cost_analysis/examples/df_r1.md.expected | Golden expected report output for DF R1 example. |
| utilities/cost_analysis/examples/df_r1.json | JSON driver for DF R1 example. |
| utilities/cost_analysis/examples/df_r1.inp | Serialized equation input for DF R1 example. |
| utilities/cost_analysis/examples/ccsd_r2.md.expected | Golden expected report output for CCSD R2 example. |
| utilities/cost_analysis/examples/ccsd_r2.json | JSON driver for CCSD R2 example (includes dump_tree). |
| utilities/cost_analysis/examples/ccsd_r2.inp | Serialized equation input for CCSD R2 example. |
| utilities/cost_analysis/examples/.gitignore | Ignores generated reports, filtered files, and dump-tree outputs. |
| utilities/cost_analysis/cost_analysis.hpp | Defines shared config/result data structures between analysis and reporting. |
| utilities/cost_analysis/cost_analysis.cpp | Implements the CLI entrypoint, JSON parsing, context setup, analysis pipeline, cache simulation, and report emission. |
| utilities/cost_analysis/CMakeLists.txt | Adds build target and CTest registrations for the new utility. |
| utilities/CMakeLists.txt | Wires utilities/cost_analysis into the utilities build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The generated timestamp changed every run; only the volatile-line stripping in run_tests.cmake kept the reference tests deterministic. Remove it (and the now-unused <chrono> include). Keep the SeQuant version/hash line, still stripped before diffing. Reference fixtures now carry a <version> placeholder instead of a frozen timestamp+hash.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cost and Factorization Analysis Utility
Adds
utilities/cost_analysis, a standalone tool that estimates the cost of evaluating a tensor equation without running it. It reads a serialized equation, optimizes and binarizes it, then writes a Markdown report on the largest intermediates, the most expensive contractions, peak storage, operation counts, and cache reuse.Inspiration is from MPQC's
SeQuantEngineclass and the several knobs it exposes.What it does
<head> = <rhs>in SeQuant serialization V1), plus context (spin basis, field, index-space sizes, factorization spaces).deserialize→ strip leading (anti)symmetrizer →flatten→optimize+binarize→ catalog the tree's internal nodes → cache simulation across all results.contractions, a Shape census by O/V/X signature, and a Cache table.
Usage
All driver knobs and their defaults are documented in the utility's
README.md.Tests
Two reference tests (
ctest -R cost_analysis_) run the tool onexamples/and diff against frozen*.md.expected.Not supported right now
AsyCostis not proto-index aware; this utility should work onceAsyCostis extended.Example report:
EOM-CCSD(2h2p) R2 (53 terms; i=10, a=38)
Terms: 53; distinct intermediates: 94; reused: 9; largest: 8.3728 MB; peak storage: 40.1953 MB.
Total operations (symbolic): 4i^2a^4 + 18i^3a^3 + 26i^4a^2 + 2ia^4 + 30i^2a^3 + 104i^3a^2 + 26i^4a + 4ia^3 + 55i^2a^2 + 4i^3a + 10*i^2a
Largest intermediates
Most expensive contractions
Shape census
Cache (gated, volatile leaf = "R", min_repeats = 2)