eval/optimize/dryrun: nested aux+PAO batching, perf-first objective, dry-run cost-profile backend, rank-general CSV moments#568
Open
evaleev wants to merge 71 commits into
Open
Conversation
…vel >= 2) Add a predict_hook to CacheManager, mirroring shaped_product_hook but observe-only: consulted at each binary Product node before materialization (self-gated on the eval trace level) to emit a "Predict" line naming the op and its predicted result footprint. This names an intermediate that exhausts memory even when it dies materializing (the post-hoc Eval line never prints for such an op). The hook is propagated into the batched scratch cache; batched products carry no shaped-product hook, so they are reported as unshaped(batched). The TA backend factory make_predict_hook() builds the hook from a TAEvalContext, reusing the shaped hook's result-outer-trange computation and a ToT average-inner-extent estimate to predict the footprint.
Add Logger::eval.heap_stats (std::function<std::string()>); when set, its (already rank-reduced) return is appended to each Eval line after rss=, and omitted entirely when unset. Lets a backend report allocator-level memory (e.g. glibc all-arena in-use vs system) right at an RSS jump, to separate live heap from retained-free heap.
Add Logger::eval.release_memory (std::function<void()>) and call it via log::release_after_op() after each freshly evaluated op in evaluate(), regardless of trace level. Lets a backend return retained-free heap to the OS between ops -- e.g. the large per-batch transients freed while building a batched DF leaf -- so they do not linger as allocator-retained free pages and inflate RSS before the next big op allocates. Local/non-collective by contract; empty hook = no-op (default).
evaleev
force-pushed
the
feature/eval-predicted-peak-trace
branch
from
5810a5c to
8e23906
Compare
The eval-time footprint estimate lifted the result's dense outer volume by an operand's average inner density (size_in_bytes / dense outer volume). That borrows the operand's fill, so it underestimates any contraction whose result is denser than its operands -- most severely the free-mu~ giant g(mu~,mu~,K).C, where the free mu~ spreads from the per-pair domain to the full range (~3x low in practice). It is also computed from materialized operand sizes, which are not what the factorizer's analytic cost model uses, so it cannot reflect or validate the tree it chose. Its one unique feature -- naming an op that OOMs before its post-hoc Eval line prints -- does not survive the real failure mode (async SIGBUS / OOM-kill), where the buffered Predict line never flushes. Remove make_predict_hook and its tot_avg_inner_factor / pred_outer_elems helpers, predict_hook_type, set_predict_hook / predict_hook(), the eval-time consultation in evaluate(), and the make_batched_scratch propagation. The post-hoc Eval line (actual sizes) and the shaped-product hook are unchanged.
Regression guard for the C60 PNO-CCSD OOM. The batched CSV-CCk residual
term
g{i_3;i_1;K} C{a_1<i_1,i_2>;mu} g{mu;mu;K} C{mu;a_3<i_2,i_3>}
factorizes, under aux(K)-only batchability, into the double-proto
intermediate I(i_1,i_2,i_3,K; a_3<i_2,i_3>, a_1<i_1,i_2>) -- two 2-occ
PNO composites sharing i_2, ~185 GB materialized -- which is what OOMs on
C60 (1800 AOs, 8 ranks, 768 GB node).
The probe pins the mechanism:
- DenseFLOPs / DensePeakSize pick a g.g-first tree that never forms the
double-proto (max_imed 1.3e11 elems);
- DensePeakSizeBatched with only K sliceable picks the double-proto
(max_imed 1.8e13) -- correctly, since with K the sole batch axis the
double-proto's K-sliced peak beats the avoiders' unsliceable mu-tilde
intermediate;
- making mu-tilde (PAO) batchable TOO flips STO back to the g.g-first
tree (max_imed 1.3e11) -- the double-proto becomes avoidable.
So the OOM root cause is the too-small batchable-index set (aux only),
not a suboptimal STO or a batching-eval bug. Guards any future change to
the batched peak objective or the batchable-index policy.
… to PeakBatchedModel
…ax, min-peak fallback) PeakBatchedModel::reconstruct now selects the root-frontier point by peak_threshold (bytes) instead of the unconditional peak_flops_tolerance band: among points whose peak*numeric_size fits peak_threshold, pick min flops (ties by lower peak); if none fit, fall back to min peak. Default peak_threshold = +infinity makes every point feasible, so min-flops wins, matching the non-batched schedule unless a finite threshold is set. peak_flops_tolerance is kept on PeakBatchedModel for source compatibility but is no longer consulted there; it still gates the unbatched DensePeakSize model. Doc comments in options.hpp/single_term.hpp updated to reflect the split. Adjusted three existing batched tests whose assertions were inherently peak-driven (not flop-driven) demonstrations, so the default +inf threshold no longer masks what they exercise: - "OSV deferral reproducer (tetramer term 3)": the persistent_only-gate probe now sets peak_threshold=1.0 to force the min-peak fallback. - "quadratic bubble: early-K integral vs late-K t.(gC)": choose() and real_config_integral() now set peak_threshold=1.0 for the same reason (this test is gated behind __OPTIMIZE__ and was verified via a separate -O1 recompile of test_optimize.cpp relinked against the Debug libraries). New test: "threshold gates batching: default (+inf) picks min-flops regardless of K_b; near-zero threshold falls back to min-peak", reusing the quadratic-bubble motif to show the divergence directly.
…s.term_batch_axes
Task 3.2 of the PNO-CCSD aux+PAO batching feature: thread the per-node
sliced-Index-sets that PeakBatchedModel::reconstruct_axes (Task 3.1)
computes out of the optimizer, so each optimized residual summand
carries which axes to slice at each of its contraction nodes.
- cost_model.hpp: add run_single_term_opt_axes, a companion to
run_single_term_opt that calls Model::reconstruct_axes instead of
reconstruct, returning {EvalSequence, node_axes} with the same nt==1/
nt==2 shortcuts (empty axes / one empty entry respectively).
- single_term.hpp: thread an optional
container::vector<container::svector<Index>>* out_axes = nullptr
through both the detail and public single_term_opt overloads. The
DensePeakSizeBatched arm calls run_single_term_opt_axes when out_axes
is set; every other objective asserts out_axes is null. Both
overloads clear *out_axes on entry so an early return (e.g. prod
with < 3 factors) never leaves it holding a previous call's data.
- optimize.cpp (opt_pure_product): declare node_axes, pass it through
only on the DensePeakSizeBatched call path, and after building the
result record (*opts.term_batch_axes)[result.get()] = node_axes when
the out-channel is set. optimize_impl/optimize() need no changes:
OptimizeOptions is threaded by reference/value-copy already, so the
shared_ptr out-channel survives unchanged.
- options.hpp: add OptimizeOptions::term_batch_axes (a
shared_ptr<unordered_map<Expr const*, per-node axes>>), default null
(no behavior change). Added the includes/forward-decl (container.hpp,
<unordered_map>, <memory>, class Expr) it needs.
Verified the RPN-to-Product alignment: single_term_opt's Product-
building loop (single_term.hpp) iterates the EvalSequence left to
right and, on each -1, pops rexpr (top of stack, most recently pushed)
then lexpr (pushed before it) to form Product{lexpr, rexpr}. Since the
sequence is emitted in the same left-first post-order in both
reconstruct and reconstruct_axes (identical build() recursion:
recurse into the lp_first-selected "first" operand, then the
"second", then append -1), the j-th -1 encountered by the loop is
exactly the j-th entry pushed into reconstruct_axes's node_axes. No
behavior change for existing callers (term_batch_axes defaults to
null); build + [optimize] unit tests pass (339 assertions, 12 cases).
Task 3.3 of the PNO-CCSD aux+PAO batching feature: consume Task 3.2's
OptimizeOptions::term_batch_axes at binarize time.
- eval_expr.hpp: add a private container::svector<Index> batch_axes_{}
member to EvalExpr with batch_axes()/set_batch_axes() accessors, and
BinarizationOptions::node_batch_axes (per-contraction-node sliced-sets,
RPN/post-order, left-first). Default-empty in both cases, so existing
callers see no behavior change.
- eval_expr.cpp: thread a std::size_t& node_counter through
impl::binarize / binarize(Sum, ...) / binarize(Product, ...). In
binarize(Product, ...)'s make_prod lambda, the tensor*tensor branch is
the genuine DP contraction node (scalar*scalar and scalar*tensor
branches are scaffolding from bare scalar factors or the trailing
scalar wrap, never counted by the optimizer): stamp
opts.node_batch_axes[node_counter] onto it when in range, then always
advance node_counter. The top-level binarize(ExprPtr, ...) entry point
owns the counter and, when opts.node_batch_axes is non-empty, asserts
the final count matches its size -- a mismatch means the optimizer's
and binarize's post-orders diverged.
Verified the alignment: single_term_opt builds every contraction as a
Flatten::No 2-factor Product, so binarize's fold_left_to_node visits
exactly one tensor*tensor node per level, in the same left-first
post-order reconstruct_axes emitted node_axes in.
- test_optimize.cpp: new [optimize][annotate] test round-trips
optimize() (DensePeakSizeBatched, forced batching via a tiny
peak_threshold, an aux Κ index shared between two factors) through
binarize(), and checks a node got annotated with the aux index that
was force-batched.
[annotate] (new test) / [optimize] (13 cases, 344 assertions) /
[EvalExpr] (81 assertions) / [EvalNode] (84 assertions) / [export]
(1955 assertions) all pass.
…h counts Confirms Task 4.2's re-entrant nesting already threads make_scope_guard unchanged into the reinstalled inner evaluator, and the outer scope_guard RAII object stays alive across the per-batch evaluate() calls that trigger that re-entry. So a backend guard that relaxes block-sparse screening scaled by its own level's batch count composes multiplicatively across nesting depth for free: net relaxation = product of batch counts over all alive levels, matching the invariant that a contribution significant over the full product of batch axes must not be screened away in any individual per-level batch cell. Documents this nesting semantics at the make_scope_guard parameter, the no_scope_guard/make_no_scope_guard default-factory doc, and the scope_guard RAII site in make_batched_custom_evaluator. Adds a structural composition test (dense TensorD has no real block-sparse screening to relax, so numeric validation is deferred to the Phase 6 end-to-end MPQC run): a custom ScopeGuardFactory whose RAII guard records construction/destruction against a shared "currently alive" stack, driven over the existing depth-2 nested batching tree. Both guards are alive simultaneously (stack depth 2) once per outer batch, and every such instance records outer_batches * inner_batches == 9, proving the multiplicative composition invariant.
sliced_footprints priced a sliced axis at min(e, batch_target_size(ix)) with no floor, while the TiledArray backend's result materialization floors the realized slice to max(target,1) (result.hpp:374). A target of 0 (a common default, e.g. MPQC's aux_target_size) made the DP model see a zero-byte footprint for that axis and never force a second axis to also slice, even though the runtime slices to a whole tile anyway. Floor the model target to >=1 so it matches the runtime in the degenerate zero-target case.
…not hardwired dense
…LOPs+exec cost, not just size)
…sidual Adds sequant::eval::dryrun::SizeRegime (per-space extents + CSV/OSV power-mean moment tables) and a dedicated unit-test OBJECT-lib group exercising it against the real C60 CSV-CCSD residual fixture (mpqc's csv_eqn_Rs.serialized). Confirms the deserialize path and the DensePeakSizeBatched + term_batch_axes + binarize round-trip mechanics end-to-end (per-summand optimize(), Product::Flatten::Yes re-flattening of the deserializer's literal nesting), plus a self-consistency memsize check. The fixture is pre-transform (no PAO/DF-aux indices), so this is infrastructure and mechanics only -- the actual PAO/K batch-axis go/no-go verdict needs a post-transform fixture and is deferred to a follow-up. See .superpowers/sdd/task-1-report.md for the full history.
…kend, and runtime replay Completes the DryRun eval backend across two fronts: 1. The real (post-transform) PAO(mu~)/DF-aux(K) batch-axis go/no-go verdict, deferred by Task 1: a [dryrun-df] case that deserializes the real post-transform CSV-CCSD doubles residual (fixture tests/unit/data/csv_ccsd_doubles_residual_df.txt, dumped from mpqc at the exact point handed to sequant::optimize()), runs the batched single-term optimizer under a C60-scale regime, and confirms the DP annotates a mu~ batch axis on the free-mu~ giant intermediate (matching the ~1.2 TB cluster-trace anchor almost exactly). 2. The DryRun eval backend itself (Tasks 2-6 of doc/dev/plans/2026-07-04-dryrun-eval-backend.md): CostModel wraps the optimizer's own memsize_counter/flops_counter/roofline_op_cost verbatim, adding only an ExtentOverrides indirection so a Result can report a runtime-realized (sliced) size rather than always the regime's nominal extent. ResultDryRun/ResultDryRunNested are zero-data Result tokens (index set + cost model only, no tensor storage) mirroring ResultTensorTAPP's structure; both concrete types share one op implementation (DryRunOps) and dispatch flat-vs-nested by content via make_dryrun_result, matching how the real engine decides tensor-of-tensor-ness. EvalExprDryRun/DryRunLeafEvaluator complete the plumbing evaluate() needs. The [dryrun-eval] harness replays the giant term's DP-annotated, binarized schedule through the REAL runtime (make_batched_custom_evaluator / evaluate<Trace::On>, with SEQUANT_EVAL_TRACE defined for this translation unit so note_working_set() fires through the batched evaluator's nested per-batch/per-member recursion, not just the outermost interception) against these zero-data tokens, reusing one BatchPolicy object for both optimize() and the runtime evaluator. On the current codebase the replay does NOT reproduce the suspected "K-sliced but mu~-full" bug: the giant's realized peak stays near 1.5 GB (a >99.9% reduction from its 1.21 TB nominal size), and the largest single materialized result (1.556 GB) matches the nominal size divided by the product of the mu~ and K batch counts (18 x 43) to within tile-rounding, i.e. both axes are genuinely nested-sliced by the runtime as currently implemented.
Add a design spec for a perf-first / peak-second single-term optimization
objective. The existing DensePeakSize model is peak-first: peak_threshold is a
hard filter over factorizations and FLOPS only breaks ties among survivors.
With PAO (mu-tilde) sliceable this structurally prefers the fully-sliceable but
FLOPS-catastrophic 4-PAO integral over the correct (gC)(gC) PPL, whose PNO-pair
leg has an irreducible peak floor above the threshold. This is the C60
PNO-CCSD OOM root cause.
The fix swaps the primary/secondary keys in the two frontier selectors
(select_root, pareto_best), keeping peak_threshold as the slice-down-to target
rather than the feasibility gate. Slicing is perf-neutral in the roofline model
(cflops computed once on unsliced sizes), so a roofline-perf primary rejects the
4-PAO and leaves slicing to the peak-second key.
Names encode the lexicographic order (Dense{Primary}{Secondary}, Space=peak,
Time=perf): rename DensePeakSize -> DenseSpaceTime (peak-first, kept as a
deprecated alias), add DenseTimeSpace (perf-first, new).
Rename DensePeakSize{,Batched} -> DenseSpaceTime{,Batched} (old names kept as
deprecated aliases sharing the same enum values, so every existing
`== DensePeakSize` guard keeps working) and add the perf-first / peak-second
duals DenseTimeSpace{,Batched}.
The peak-first model treats peak_threshold as a hard filter over
factorizations, so with PAO (mu-tilde) sliceable it structurally prefers the
fully-sliceable but FLOPS-catastrophic 4-PAO integral over the correct (gC)(gC)
particle-ladder. The perf-first model adds a `perf_first` flag to PeakModel /
PeakBatchedModel; when set, the root-frontier selector sorts by (flops, then
peak) instead of (peak, then flops). Because pareto_insert keeps one min-peak
point per distinct flops value, this both picks the cheapest factorization and
takes its fully-sliced realization, and peak_threshold is no longer consulted
as a feasibility gate (it can no longer force the 4-PAO). The Pareto frontier,
relax(), roofline cost, and the pareto_best test helper are unchanged.
single_term_opt routes DenseTimeSpace{,Batched} to the peak models with
perf_first set; optimize() gains dispatch arms for both new values.
Tests: DenseTimeSpaceBatched reaches the flop-optimal (gC)(gC) factorization at
both +inf and a near-zero peak_threshold (unlike peak-first strict, which pays
the ladder), and the enum aliases compare equal.
Add the [dryrun-perf] test plus the post-transform DryRun harness apparatus
([dryrun-probe]/[dryrun-df]/[dryrun-eval]) used to diagnose the C60 PNO-CCSD
OOM. [dryrun-perf] optimizes the real C60 giant term (index 38) under both the
peak-first (DenseSpaceTimeBatched) and perf-first (DenseTimeSpaceBatched)
objectives at the faithful config (occ=120, PNO=42, OSV=310, mu-tilde=1800,
aux=4320, pao_ts=256, aux_ts=72, peak_threshold=40 GB) and asserts the
factorization the DP picks.
Result: peak-first forms the fully-sliceable 4-PAO AO integral (4 free
mu-tilde, sliced to 34 GB, under the 40 GB threshold, so it survives the hard
filter despite catastrophic flops) -- the C60 pathology. Perf-first forms no
4-PAO (max 2 free mu-tilde); its largest giant is the correct (gC)(gC)
intermediate {mu-tilde, a<i,i>, K} at 89 GB, far below the 627-769 GB that
OOM'd the real run and below node memory. The fix is validated in-harness.
Extend [dryrun-perf] to also REPLAY the zero-data schedule (both objectives)
through the real eval loop + real CacheManager and report
cache.working_set_hwmark() alongside the largest single transient (result=) and
its full trace line.
Two findings, now documented in the test:
(1) The outer working_set_hwmark is NOT the whole peak under batching. The
batched custom evaluator runs each batch against a SEPARATE scratch cache
(make_batched_scratch, eval.hpp:1386), so the peak-first 4-PAO giant's per-op
hw= reaches ~38.9 GB INSIDE that scratch while the outer accessor reports
only ~0.2 GB. The outer accessor tracks cached (cross-batch) residency, not
batched-inner transients.
(2) The runtime Result sizing is not moment-aware. The perf-first largest
transient = 358.47 GB = 120^2*42^4*8 (occ^2*PNO^4) EXACTLY = the twin-PNO
size_in_bytes() artifact; the DP cost model's moment-aware size (~89 GB, via
inner_pow) is the real number.
Conclusion: the moment-aware DP peak model is the reliable predictor today; the
runtime replay hwmark is not, until scratch hwmarks propagate and the Result
sizing is moment-aware.
… + batch_axes per node)
…d sweep
Make the [dryrun-df] C60 harness a controlled experiment for the PNO-CCSD
schedule study:
- df_regime array overload takes the real per-nonnull-cluster power means
M_1..M_4 (heavy tail) as measured by mpqc's PaoPnoRMP2, instead of the
constant-domain scalar (M_k = mean for all k, which under-sizes
multi-composite intermediates).
- env sweeps on [dryrun-df]: SEQUANT_UT_DRYRUN_OBJ (perf=dense_time_space vs
peak=dense_space_time), SEQUANT_UT_DRYRUN_{PNO,OSV}_M2..M4 (real heavy tail),
SEQUANT_UT_DRYRUN_PEAK_THR_GB (batching peak budget). All default to the prior
constant-M1 / peak-first / 40 GB behavior, so absent env vars reproduce the
old test exactly.
- report per-term and total nominal schedule FLOPS alongside the realized peak,
so the flops-vs-peak tradeoff between factorizations is visible (the 4-PAO
fully-sliced integral is memory-cheap but flops-heavy; only this surfaces it).
Used to establish: perf-first (dense_time_space) ignores peak_threshold and
forms the ~1.2 TB free-mu~ giant (C60 OOM); peak-first at a realistic threshold
slices mu~ and fits. Also surfaced a threshold non-monotonicity in the
gated select_root worth a separate look.
… cost) Add a SEQUANT_SELROOT_DEBUG-gated stderr print in PeakBatchedModel::select_root reporting the DP's OWN chosen frontier point cost (.flops = the roofline exec cost, volatile-weighted -- the true objective), its peak, whether any point was feasible, the frontier size, and the global-min .flops over the whole frontier. This is the tool for calibrating batch:peak_threshold on real systems: summing chosen_flops across a residual's terms at several thresholds shows the threshold-vs-cost curve directly (in the DP's own metric, not a post-hoc proxy). It is what disambiguated the C60 study -- the chosen cost is monotone in the threshold, and a too-tight threshold (40 GB) sits ~38x above the global optimum because every low-cost schedule is peak-infeasible. Zero overhead when the env var is unset.
…f sweep The [dryrun-df] sweep reported only nominal contraction flops, which are batch-independent and can move non-monotonically with the peak threshold. Add a per-term and overall roofline exec-cost accumulation matching the DP's actual optimization axis (max(flops, mb*max(traffic, prefac*flops/ sqrt(fastmem/tiles))), mb=200) so the sweep prints the quantity the optimizer minimizes, not just raw flops.
The batched cost model assumed WORK PARITY: slicing an axis shrinks peak but never changes flops. That holds for axes a node CARRIES (slicing partitions the work, total preserved), but not for axes an ancestor slices that the node's result does NOT carry -- there the batched evaluator re-executes the node once per tile (across-batch recompute; within-batch sharing is cached, per eval.hpp "replays the build of every compatible persistent final"). Work-parity under-costs schedules that recompute across many sliced axes. Add PeakBatchedModel::charge_batch_recompute: when set, a node at ancestor-sliced-set B is charged nbatch(b)=ceil(extent/target) for each b in B not open in the node. Context now carries per-axis nbatch. Default false preserves the historical cost model exactly; wired in single_term via the SEQUANT_CHARGE_RECOMPUTE env var for validation (TODO: promote to an OptimizeOptions/BatchPolicy field). Finding on the C60 giant: the competitive schedules slice axes they carry (partitioning), so the recompute charge is small and does not by itself redirect selection -- the 4-PAO's flops were already accounted correctly. The C60 OOM is a peak/feasibility problem (the giant does not fit), fixed by making mu~ batchable plus a realistic peak threshold, not by this charge. The charge is a faithfulness fix, kept off by default.
The batch recomputation cost is a real cost of batching, so the cost model should always reflect it. Flip PeakBatchedModel::charge_batch_recompute to default true and drop the SEQUANT_CHARGE_RECOMPUTE env gating in single_term (the flag remains as a programmatic escape hatch for work- parity comparison). No SeQuant unit-test regressions: [optimize] 365 assertions/14 cases and the tagged [dryrun] unit tests stay green.
test_eval_dryrun.cpp had accreted a Phase-1 investigation layer built
against the PRE-transform residual (csv_ccsd_residual.txt: 4-index g, no DF
aux, generic base-virtual 'a' as a stand-in for mu~) whose stated purpose
was to exercise the DP-inspection plumbing until a post-transform fixture
existed. That fixture (csv_ccsd_doubles_residual_df.txt, real mu~ + K)
exists and the post-transform [dryrun-df]/[dryrun-perf]/[dryrun-eval] cases
now do the real batch-axis verdict, so the pre-transform layer is dead
weight. Remove:
- TEST 'DP-inspection mechanics on pre-transform residual' (explicitly a
stand-in, verdict-pending case, ~185 lines);
- TEST 'residual fixture deserializes' (deserialized the old fixture);
- TEST 'harness sanity: forced threshold annotates a batch axis'
(synthetic-network plumbing check, now covered by the real cases);
- dead helpers split_residual_members / is_stand_in_batchable;
- the csv_ccsd_residual.txt fixture (no remaining references).
Rename the stand-in c60_regime -> probe_regime (its only remaining users
are the SizeRegime/memsize contract unit tests) and rewrite the stale file
header. Kept: the SizeRegime/CostModel/Result unit tests and every
post-transform batched-schedule case. Fast dryrun groups green (42
assertions/15 cases); TU compiles clean.
…NO legs
Add env knobs to the perf-first-vs-peak-first case so it can answer 'is the
4-PNO PPL integral formed?' for any summand at any budget:
- SEQUANT_UT_DRYRUN_LIST_TERMS=1 -- print every summand (to_latex) to locate
the PPL/ladder term (term 38: two PAO-PAO DF g's + four C's + t^{ij}_{a3a4});
- SEQUANT_UT_DRYRUN_TERM=N -- select the summand (default 38);
- SEQUANT_UT_DRYRUN_PEAK_THR_GB -- peak_threshold in GB (default 40, was
hardcoded) so the real job budget (600) can be reproduced;
- the PERF_TREE dump now also reports PNO/OSV composite free-leg counts, so a
4-free-PNO node (the held-whole 4-PNO integral) is unmistakable.
Confirms: at 600 GB peak-first keeps max PNO-per-node = 2 (contracts t early,
never forms the 4-PNO integral), while perf-first forms the {a1 a2 a3 a4} node
(the 4-PNO integral, ~pno^4 per pair). Defaults unchanged; [dryrun-perf] green
(13 assertions).
…lat 42 The perf-first-vs-peak-first case hardwired the SCALAR df_regime(pno=42, osv=310) -- a flat constant domain (M_k=42 for all k) -- so the 4-PNO PPL node it reports was sized occ^2*42^4 = 387 GB, an under-count. Add named constants kC60PnoM/kC60OsvM = the real heavy-tailed power means measured by mpqc PaoPnoRMP2 (jobs 617609/617653/617809: PNO M_1..M_4 = 48.60/54.34/ 59.63/64.35, OSV = 206.48/216.66/226.15/234.51) and feed them via the FAITHFUL array df_regime overload. The 4-PNO node now sizes occ^2*M_4^4 ~= 2.0 TB (dense occ^2 pairs; ~0.86 TB at the screened ~6300 CC pairs) -- faithfully showing why perf-first OOMs C60. Peak-band assertion updated 1..4 TB; [dryrun-perf] green (13 assertions).
…oments Three cleanups in one: 1. DRY the problem sizes. Every post-transform test repeated the C60 literals (df_regime(1800,4320,120,...)) at its own call site. Introduce a constexpr ProblemSize struct (extents + PNO/OSV power means) and one instance kC60_pVDZF12, plus a df_regime(ProblemSize) overload; every fixed call site is now df_regime(kC60_pVDZF12). A moment re-measurement is a one-line edit. Remove the now-unused scalar df_regime(...,double,double) overload. 2. Correct the moments. The values 48.60/54.34/59.63/64.35 (PNO) and 206/234 (OSV) came from job 617653, which was mis-configured with an aug-cc-pVTZ OBS. The true cc-pVDZ-F12 moments (job 617809) are PNO 42.03/46.04/49.77/ 53.15 and OSV 148.25/155.04/161.34/166.86. With these the perf-first 4-PNO PPL node measures occ^2*M_4^4 = 954 GB (was mis-stated 2 TB at the wrong M_4). All post-transform tests (incl. [dryrun-df] env defaults) now use the real moments, not the flat 42/310 stand-in. 3. Rename [dryrun-perf] -> [dryrun-objective]. "perf" only named one of the two objectives it compares; the test's point is that the objective function DETERMINES the factorization (perf-first forms the 4-PNO integral, peak-first does not). Title updated to say so. All 21 dryrun cases green (120 assertions).
Two overlapping whole-residual harnesses, neither asserting correctness: [dryrun-trace] replayed the forest through the shared cache and wrote a per-op trace file; [dryrun-df] prints the per-term mu~/K batch-axis escape verdict. The trace replay is already covered by the [dryrun][cost_profile] unit tests and the trace file is a regenerable one-off, so cut [dryrun-trace] entirely. Keep [dryrun-df]'s per-term verdict (uniquely useful C60 diagnostic, with the env sweep knobs) but mark it hidden ([.]) since it is a dev sweep, not a regression test -- select it explicitly with "[dryrun-df]". The standing tests are now [dryrun-objective] (comparative) + [dryrun-eval] (runtime replay) + the SizeRegime/CostModel/Result unit tests + the [peak]/[cost_profile]/[cache] infra tests. Visible dryrun suite green (120 assertions/21 cases).
Sweeps all 55 C60 residual terms under both objectives and reports the summed-flops ratio (perf-first/peak-first) and max modelled peak per objective -- the decision input for whether enabling dense_time_space (perf-first) at scale is worth it. Test-only; no production change. Result: ratio 0.142 (perf-first ~7x fewer flops), max peak 49.5 vs 5862 GB.
Replace the code-recursive descent in evaluate(node, le, cache) -- the only evaluate overload with unbounded recursion (the layout/multi-root overloads just wrap or loop over it) -- with an explicit std::deque work stack. Depth is now bounded by the heap, not the C++ call stack, so a deep tree (a Sum or product chain with many operands) no longer risks a stack overflow in the traversal. The per-node cache key is EvalExpr's stored O(1) hash, so nothing in the walk recurses. The rewrite preserves behavior byte-identically: - Checked cache wrapper: a hit returns the phase-applied cached pointer; a miss on a mapped node schedules a store once computed (the store_after flag replaces the recursive evaluate<..., Unchecked> re-entry). - Custom-evaluator interception is consulted when a frame is first visited and a non-null result short-circuits the subtree (children never pushed) -- the subtree pruning batched eval relies on is unchanged. - leaf / Adjoint / Sum / Product dispatch, the shaped-product hook, de-nesting, canonicalization-phase multiplication, eager per-op release, and every trace log site (MultByPhase, custom-eval, leaf, binary, cache access/store) are carried over unchanged. Verified against the eval oracle ([eval_tapp], [dryrun-eval] incl. the batched custom-evaluator giant-term replay, [dryrun-leaf/result/nested/costmodel], [EvalNode], [EvalExpr]): 247 assertions in 16 cases, identical to the pre-refactor baseline. Note: this makes the *traversal* stack-safe; binarize/EvalExpr construction and FullBinaryNode's (unique_ptr-owned) destructor remain code-recursive and would overflow first on a single very deep tree. Full-pipeline stack-safety for one arbitrarily deep tree would need those addressed too; in current use the residual is a vector of shallow summand trees.
…tive FullBinaryNode owns its children by unique_ptr, so the implicitly-generated destructor recursed to the tree's depth; deep_copy() and the free function transform_node() likewise recursed. On a deep tree (a long contraction chain or a nested Sum) these overflowed the C++ call stack at depth ~thousands, which is why building/copying/destroying such a tree crashed before evaluation even ran. Replace all three with explicit-stack iterative implementations: - ~FullBinaryNode() dismantles the subtree via a work list, detaching each node's children before it is destroyed so every node destruction is O(1). - deep_copy() and transform_node() do an iterative post-order build with an explicit frame stack (std::deque, so a top-frame reference survives push_back). transform_node applies its data map post-order; as a pure map its application order is immaterial, and its only callers (typed binarize<T> and the TA to_ta_node helper) are order-insensitive. The tree fold (fold_left_to_node / the range-accumulate ctor) and the tree visitors (detail::visit, parent-pointer based) were already iterative. size(), operator==, and digraph/tikz remain recursive but are not on the deep-tree build/eval/destroy path. Adds a [FullBinaryNode][stack-safety] test that builds, copies, transform_nodes, and destroys a depth-200000 tree (all O(N)); pre-refactor this overflowed the stack. Full FullBinaryNode + eval suites remain green.
…ort giant under occ-slice
Reconcile the predicted-peak / batching / dry-run work with master's merged #576 (outer-product pruning + build_context skip + fast_flops), #575/#574 (iterative evaluate()/binary_node), and TNv3 high-order aux work. Conflict resolutions: - options.hpp: CostParams and OptimizeOptions carry BOTH the branch's peak_threshold / term_batch_axes and master's prune_outer_products. - single_term.hpp: keep the perf-first objective dispatch (perf_first, DenseTimeSpace/DenseTimeSpaceBatched, peak_threshold, out_axes) and thread master's prune_outer_products into every model. - cost_model.hpp: PeakModel / PeakBatchedModel keep the branch's perf-first members (perf_first, peak_threshold, numeric_size, charge_batch_recompute) and gain master's prune_outer_products; build_context bodies auto-merged so the pruning skip + fast_flops precompute coexist with the perf-first tables. - optimize.cpp: CostParams init passes both peak_threshold and prune_outer_products. - eval.hpp: keep the branch's malloc_trim release_after_op() hook at each op. - test_optimize.cpp: keep both the branch's threshold/perf-first tests and master's pruning + fast-flops-parity tests. The branch's own (superseded) outer-product-pruning commits were dropped before this merge since #576 supersedes them; the predict-hook add/remove experiment nets to no hook, matching master.
The forest-over-external-axis nesting test built its external Hadamard spectator as a bare occupied index shared among three tensor slots (g, h, p) plus the result. Master's TensorNetworkV3::canonicalize_slots deliberately rejects a non-auxiliary index shared among >2 tensor slots (it has no well-defined bra/ket slot type), so eval_node threw once this branch merged master's newer TNv3. Switch the spectator to an auxiliary index (x_9): a high-order aux hyperindex carried into the result IS supported (the same case the eval_with_tiledarray test exercises), the arrays stay flat so the intra-term batched evaluator runs on its supported path, and the forest + nested-batching mechanism under test is axis-agnostic. A second inner-aux predicate keeps the contracted batch axis (x_1) distinct from the spectator. Occupied-external recognition and sizing are covered separately at the DP/cost level.
is_valid recognized Constant, Variable, Tensor, Product and Sum but fell through to a hard assertion for any other expression type. Power (base^ exponent) is a legitimate expression -- it appears, for example, in CC/EOM residual expressions as an inverse-denominator factor -- so a consumer that validates such an expression (e.g. MPQC's process_for_evaluation) aborts in assertion-enabled builds and, with the assertion elided in release, silently treats a Power-bearing expression as valid only by luck. Add a Power case: a Power is valid iff its base is valid (the exponent is a rational and is always well-formed). Power is atomic, so the base is not reached by the children loop and is validated explicitly here.
…l spaces is_pure_occupied/is_pure_unoccupied/contains_occupied/contains_unoccupied reduced an index's quantum numbers to physical-particle (spin) attributes and then looked up the vacuum-(un)occupied space at those qns via a throwing retrieve. For an index whose reduced qns has no registered occupied space this threw instead of answering the occupancy question. Two cases motivated this: - Physical spaces (AO, PAO) span real particle states and are spin-independent, not spin-less; they must carry the convention's spin_any in the physical sector. add_ao_spaces/add_pao_spaces registered them with only their LCAO trait bit (spin bits empty), so physical_particle_attributes() reported no spin and the occupancy lookup queried a spin flavor with no occupied space. Fix: add_ao_spaces/add_pao_spaces now take an explicit spin_any parameter (Spin::null for SpinConvention::Legacy, else Spin::any, matching make_*_spaces) and register the AO/PAO spaces with spin_any | trait. - Non-physical auxiliary spaces (density-fitting, batching) legitimately carry no spin; asking whether they are occupied must answer false, not throw. The four occupancy predicates now route through non-throwing helpers (vacuum_occupied_type_or_null / complete_type_or_null / vacuum_unoccupied_type_or_null) that return a null type when no space is registered at the reduced qns. Byte-identical for physical inputs. Update unit-test callers of add_ao_spaces/add_pao_spaces to pass the spin_any matching each test's registry convention.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eval/optimize/dry-run infrastructure for costing, sizing, and batching CSV-CCk / PNO-CCSD factorizations, motivated by the C60 PNO-CCSD OOM (the dense free-μ̃
(g·C)(g·C)giant). This branch grew well past its original "observe-only predicted-footprint hook" scope (that hook has since been removed — see below). Paired MPQC PR: ValeevGroup/mpqc4#783.What this delivers
1. Nested multi-axis (aux + PAO + occupied) batching in the evaluator
EvalExpr.batch_axes_:binarizestamps per-node sliced-sets; the batched evaluator prefers the per-node annotated batch axis.optimize()recognizes such occupied protoindices as batchable and emits the choice as a forest-levelExternalBatchAxissignal (domain-neutralBatchPolicyconfig). This lets PNO-CCSD batch over occupieds, not just aux/PAO, to bound the dense free-mu-tilde(g.C)(g.C)giant.OptimizeOptions.term_batch_axessurfaces per-node sliced-sets per summand in RPN order.2. Threshold-gated batched selection in
optimize()PeakBatchedModel/reconstructed_batched_peakoracle prices the batch contribution buffer;peak_threshold(bytes) plumbed throughBatchPolicy/CostParams.accumulation_factorallowed for m≥2 (nested), gated by the DP==oracle identity. Sliced-axis model target floored to ≥1 to match the runtime.3. Perf-first
DenseTimeSpace{,Batched}single-term objective4. Dry-run cost-profile eval backend
Resultbackend +CostModelthat replays a factorized IR and reports memsize + FLOPs + exec cost (a pluggable sizing oracle; dense by default), not just size.CostProfilestruct + reusablecost_profile()entry point;[dryrun-perf]/[dryrun-trace]routed through it. Opt-in scratch-fold peak sink for a faithful batched-replay peak; gated cache with batchable-axis veto and per-term reset.SizeRegime: per-space extents + per-rank CSV moment tables (power means). Post-transform (μ̃ + Κ) C60 residual fixtures and DP-inspection mechanics; power-mean sizing contract locked with a regression test.inner_powdispatches by cluster rank (1→OSV, 2→PNO, ≥3→acsv_moment_by_ranktable if present, else the PNO-table fallback), so CSV-CCSDT triples are sized by their own domain. Pairs with the MPQC rank-generalCSVSizeMoments.5. Index-space registry: occupancy predicates robust to non-physical spaces
Enabling the occupied-spectator scan in
optimize()against a real CSV/PNO registry surfaced a latent registry bug.is_pure_occupied/is_pure_unoccupied/contains_occupied/contains_unoccupiedreduced an index's quantum numbers to physical-particle (spin) attributes and then looked up the vacuum-(un)occupied space via a throwing retrieve, so an index whose reduced qns had no registered occupied space threw instead of answering the occupancy question. Two coordinated fixes:spin_anyin the physical (spin) sector alongside their LCAO trait bit.add_ao_spaces/add_pao_spacespreviously registered them with only the trait bit (spin bits empty), sophysical_particle_attributes()reported no spin and the occupancy lookup queried a spin flavor with no occupied space. These now take an explicitspin_anyparameter (Spin::nullforSpinConvention::Legacy, elseSpin::any, matchingmake_*_spaces) and registerspin_any | trait.vacuum_occupied_type_or_null/complete_type_or_null/vacuum_unoccupied_type_or_null) that return a null type when no space is registered at the reduced qns. Byte-identical for physical inputs.Unit-test callers of
add_ao_spaces/add_pao_spaceswere updated to pass thespin_anymatching each test's registry convention (Default/None →Spin::any, Legacy →Spin::null).6. Fixes / misc
tot_inner_rank.CostToken/DryRunfor clarity; reframe the spec as cost-model replay.Note on history
The original observe-only predicted-footprint (predict) hook on
CacheManagerwas added and then removed within this branch (commitsa19dbfb5f→bc5bd6cb5); it nets to zero. The dry-run cost-profile backend above is its principled replacement — it predicts the whole factorized IR up front rather than per-op during materialization. This PR's title/body have been updated accordingly.Validation
[dryrun]/[dryrun-df]/[dryrun-perf]/[optimize]suites pass; new rank-general dispatch regression test passes. The registry fix keeps the full space/mbpt suite green ([elements],[mbpt],[mbpt/cc],index_space,spin,[algorithms]incl. the Legacy-convention PAO canonicalization test) and[optimize](which exercises the PAO+DFcompute_external_batch_axisemission path); MPQChe10-csv-cck-2-paocorrelation energy is unchanged vs the pre-fix binary to ~1e-13. Branch base ismaster.