Skip to content

FE-1412: Scale the Python client's timeout for seeded trials - #9229

Merged
kube merged 1 commit into
mainfrom
cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service
Aug 22, 2026
Merged

FE-1412: Scale the Python client's timeout for seeded trials#9229
kube merged 1 commit into
mainfrom
cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service

Conversation

@kube

@kube kube commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

🌟 What is the purpose of this PR?

A trial's seeded runs execute sequentially in the CLI, so one optimization.evaluate may take seedsPerTrial × the single-run time. The client's fixed 240 s response deadline must scale with it.

FE-1468 (#9262) has merged; this PR now sits at the bottom of stack #9280 on main, with FE-1470 (#9278) above.

🔗 Related links

  • FE-1412 (internal): this PR
  • FE-1408 (internal): parent — seeded trials in the CLI

🔍 What does this change?

In @local/petrinaut-python:

  • describe() reads the typed study.seedsPerTrial, rejects values outside 1–100 as a protocol error, and multiplies the per-response deadline by it.
  • evaluate() describes first when the session has not been described, so the scaled deadline is in place before the first trial.
  • A new end-to-end test drives a two-seed trial against the built CLI and asserts the derived seed sequence [42, 1013904268], the per-seed replicates, and the mean objective.

The seeded runs are sequential, so the image needs no worker permissions or pool caps.

Review fixes. The hand-rolled MAX_SEEDS_PER_TRIAL constant and its range check are deleted: FE-1468's schema bounds seedsPerTrial to 1–100, so an out-of-range value already fails model validation in _validated, which closes the session and raises PetrinautProtocolError — the same observable behaviour with one owner for the bound.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • modifies workspaces but not a publishable library.

📜 Does this require a change to the docs?

The changes in this PR:

  • require changes to docs which are made as part of this PR: the bindings and optimizer READMEs document the scaled deadline.

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph.

🛡 What tests cover this?

  • Bindings (28 tests): timeout scaling from the described seedsPerTrial, rejection of out-of-range values, and the two-seed e2e against the built CLI.
  • apps/petrinaut-opt: 75 tests, unchanged.

❓ How to test this?

turbo run test:unit --filter @local/petrinaut-python --filter @apps/petrinaut-opt

🤖 Generated with Claude Code

@vercel

vercel Bot commented Aug 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview Aug 21, 2026 3:20pm
petrinaut Ready Ready Preview Aug 21, 2026 3:20pm
petrinaut-docs Ready Ready Preview Aug 21, 2026 3:20pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Aug 21, 2026 3:20pm

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests area/apps labels Aug 16, 2026
@kube kube self-assigned this Aug 16, 2026
@kube
kube force-pushed the cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service branch from 3eda58b to 63d0a20 Compare August 17, 2026 23:51
@kube
kube force-pushed the cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service branch from 63d0a20 to 7007b77 Compare August 17, 2026 23:56
@kube kube changed the title FE-1412: Support seeded trials in the Python client and optimizer image FE-1412: Scale the Python client's timeout for seeded trials Aug 17, 2026
@kube
kube marked this pull request as ready for review August 17, 2026 23:56
Copilot AI balanced review requested due to automatic review settings August 17, 2026 23:56
@cursor

cursor Bot commented Aug 17, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes how long the Python client waits for CLI evaluations (up to 100× the base 240s deadline). Wrong scaling could hang studies or kill valid multi-seed trials.

Overview
The Python bindings now multiply the protocol read deadline by the study’s seedsPerTrial after describe(), because one evaluate can run that many simulations sequentially.

If evaluate() is called first, it describes once so the scaled timeout is in place before the trial. Out-of-range seedsPerTrial still fails via the existing pydantic schema (1–100). Docs note that petrinaut-opt ignores per-seed replicates and uses the mean objective.

Tests cover timeout scaling, implicit describe-on-evaluate, invalid seeds, and a two-seed e2e against the real CLI (derived seeds [42, 1013904268]).

Reviewed by Cursor Bugbot for commit d4126df. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Scales Python optimization response timeouts for sequential seeded trials and documents the behavior.

Changes:

  • Validates seedsPerTrial and scales response deadlines.
  • Adds unit and CLI end-to-end coverage.
  • Documents seeded execution and timeout behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
session.py Stores the base response timeout.
optimization.py Validates seed count and scales timeouts.
test_optimization_session.py Tests scaling and invalid values.
test_e2e_cli.py Tests a two-seed CLI trial.
Python README.md Documents scaled deadlines.
Optimizer README.md Documents sequential seeded trials.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libs/@local/petrinaut-python/src/petrinaut/optimization.py
@kube
kube force-pushed the cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service branch from bc69cf3 to b8b924e Compare August 18, 2026 23:19
@kube
kube force-pushed the cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service branch from b8b924e to c872ad2 Compare August 18, 2026 23:37
@kube
kube force-pushed the cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service branch from c872ad2 to bd52c92 Compare August 19, 2026 09:03
@kube
kube force-pushed the cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service branch from bd52c92 to 454b48b Compare August 19, 2026 09:40
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (cf/fe-1468-python-pydantic-models-from-cli-schemas@ecae501). Learn more about missing BASE report.

Additional details and impacted files
@@                                  Coverage Diff                                  @@
##             cf/fe-1468-python-pydantic-models-from-cli-schemas    #9229   +/-   ##
=====================================================================================
  Coverage                                                      ?   59.63%           
=====================================================================================
  Files                                                         ?     1421           
  Lines                                                         ?   138778           
  Branches                                                      ?     6557           
=====================================================================================
  Hits                                                          ?    82756           
  Misses                                                        ?    54958           
  Partials                                                      ?     1064           
Flag Coverage Δ
apps.hash-api 13.98% <ø> (?)
rust.harpc-wire-protocol 92.23% <ø> (?)
rust.hash-codec 72.76% <ø> (?)
rust.hash-graph-authorization 62.59% <ø> (?)
rust.hash-graph-validation 84.71% <ø> (?)
rust.hashql-ast 89.63% <ø> (?)
rust.hashql-mir 87.92% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Aug 21, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 15.38%

❌ 2 regressed benchmarks
✅ 96 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
bit_matrix/dense/iter_row[64] 140.8 ns 170 ns -17.16%
bit_matrix/dense/iter_row[200] 185.8 ns 215 ns -13.57%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cf/fe-1412-python-client-support-seeded-trials-in-the-optimizer-service (279a4bc) with cf/fe-1270-create-python-bindings-to-petrinaut-core (2009728)

Open in CodSpeed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$26.0 \mathrm{ms} \pm 298 \mathrm{μs}\left({\color{gray}1.83 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.42 \mathrm{ms} \pm 14.4 \mathrm{μs}\left({\color{gray}-2.860 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$13.3 \mathrm{ms} \pm 121 \mathrm{μs}\left({\color{red}7.08 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$44.2 \mathrm{ms} \pm 393 \mathrm{μs}\left({\color{gray}2.45 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.7 \mathrm{ms} \pm 135 \mathrm{μs}\left({\color{lightgreen}-5.009 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$24.7 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{gray}2.31 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$27.3 \mathrm{ms} \pm 199 \mathrm{μs}\left({\color{gray}2.07 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.80 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}-1.649 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$13.5 \mathrm{ms} \pm 105 \mathrm{μs}\left({\color{gray}1.09 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.75 \mathrm{ms} \pm 23.2 \mathrm{μs}\left({\color{gray}0.870 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.02 \mathrm{ms} \pm 15.2 \mathrm{μs}\left({\color{gray}0.204 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$3.42 \mathrm{ms} \pm 23.8 \mathrm{μs}\left({\color{gray}1.76 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$5.11 \mathrm{ms} \pm 31.0 \mathrm{μs}\left({\color{gray}0.785 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.59 \mathrm{ms} \pm 25.9 \mathrm{μs}\left({\color{gray}-0.090 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$4.11 \mathrm{ms} \pm 20.7 \mathrm{μs}\left({\color{gray}-0.747 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.37 \mathrm{ms} \pm 28.4 \mathrm{μs}\left({\color{gray}1.19 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.50 \mathrm{ms} \pm 25.7 \mathrm{μs}\left({\color{gray}0.920 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$4.10 \mathrm{ms} \pm 23.2 \mathrm{μs}\left({\color{gray}0.608 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.70 \mathrm{ms} \pm 15.3 \mathrm{μs}\left({\color{gray}0.319 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.54 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}-0.654 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$2.67 \mathrm{ms} \pm 13.0 \mathrm{μs}\left({\color{gray}-0.387 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.97 \mathrm{ms} \pm 17.7 \mathrm{μs}\left({\color{gray}0.232 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.75 \mathrm{ms} \pm 13.7 \mathrm{μs}\left({\color{gray}0.316 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.93 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}-0.277 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$3.02 \mathrm{ms} \pm 16.9 \mathrm{μs}\left({\color{gray}-1.109 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.76 \mathrm{ms} \pm 16.1 \mathrm{μs}\left({\color{gray}-0.347 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$2.93 \mathrm{ms} \pm 27.6 \mathrm{μs}\left({\color{gray}-2.428 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.42 \mathrm{ms} \pm 22.4 \mathrm{μs}\left({\color{gray}0.373 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.99 \mathrm{ms} \pm 18.9 \mathrm{μs}\left({\color{gray}-1.171 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$3.23 \mathrm{ms} \pm 19.4 \mathrm{μs}\left({\color{gray}-2.576 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.37 \mathrm{ms} \pm 23.4 \mathrm{μs}\left({\color{gray}1.01 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.00 \mathrm{ms} \pm 17.2 \mathrm{μs}\left({\color{gray}0.359 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.27 \mathrm{ms} \pm 23.9 \mathrm{μs}\left({\color{gray}-0.129 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$43.7 \mathrm{ms} \pm 245 \mathrm{μs}\left({\color{gray}2.13 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$33.9 \mathrm{ms} \pm 185 \mathrm{μs}\left({\color{gray}-0.850 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$36.6 \mathrm{ms} \pm 218 \mathrm{μs}\left({\color{gray}0.383 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$47.3 \mathrm{ms} \pm 1.34 \mathrm{ms}\left({\color{gray}4.56 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$41.9 \mathrm{ms} \pm 221 \mathrm{μs}\left({\color{gray}-1.026 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$50.0 \mathrm{ms} \pm 308 \mathrm{μs}\left({\color{gray}0.152 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$40.6 \mathrm{ms} \pm 300 \mathrm{μs}\left({\color{gray}-0.425 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$92.8 \mathrm{ms} \pm 655 \mathrm{μs}\left({\color{gray}0.286 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$34.1 \mathrm{ms} \pm 206 \mathrm{μs}\left({\color{lightgreen}-36.660 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$275 \mathrm{ms} \pm 825 \mathrm{μs}\left({\color{lightgreen}-10.571 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$11.0 \mathrm{ms} \pm 55.2 \mathrm{μs}\left({\color{gray}0.638 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$11.2 \mathrm{ms} \pm 67.5 \mathrm{μs}\left({\color{gray}-0.192 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$11.1 \mathrm{ms} \pm 59.1 \mathrm{μs}\left({\color{gray}-0.051 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$11.1 \mathrm{ms} \pm 72.0 \mathrm{μs}\left({\color{gray}0.696 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$11.0 \mathrm{ms} \pm 57.6 \mathrm{μs}\left({\color{gray}-0.145 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$11.0 \mathrm{ms} \pm 64.8 \mathrm{μs}\left({\color{gray}-0.113 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$11.2 \mathrm{ms} \pm 68.8 \mathrm{μs}\left({\color{gray}0.699 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$11.2 \mathrm{ms} \pm 57.0 \mathrm{μs}\left({\color{gray}-0.046 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$11.2 \mathrm{ms} \pm 83.9 \mathrm{μs}\left({\color{gray}-0.423 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$11.4 \mathrm{ms} \pm 63.2 \mathrm{μs}\left({\color{gray}-1.079 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$11.8 \mathrm{ms} \pm 104 \mathrm{μs}\left({\color{gray}1.52 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$11.5 \mathrm{ms} \pm 63.8 \mathrm{μs}\left({\color{gray}0.753 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$11.4 \mathrm{ms} \pm 63.3 \mathrm{μs}\left({\color{gray}-0.394 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$11.5 \mathrm{ms} \pm 62.7 \mathrm{μs}\left({\color{gray}0.597 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$11.6 \mathrm{ms} \pm 68.4 \mathrm{μs}\left({\color{gray}0.363 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$11.5 \mathrm{ms} \pm 66.6 \mathrm{μs}\left({\color{gray}0.050 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$11.5 \mathrm{ms} \pm 53.9 \mathrm{μs}\left({\color{gray}-0.332 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$11.5 \mathrm{ms} \pm 60.6 \mathrm{μs}\left({\color{gray}-0.759 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$11.5 \mathrm{ms} \pm 77.9 \mathrm{μs}\left({\color{gray}0.668 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.53 \mathrm{ms} \pm 51.7 \mathrm{μs}\left({\color{gray}-0.047 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$59.8 \mathrm{ms} \pm 586 \mathrm{μs}\left({\color{red}6.55 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$112 \mathrm{ms} \pm 895 \mathrm{μs}\left({\color{gray}2.93 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$67.1 \mathrm{ms} \pm 556 \mathrm{μs}\left({\color{red}5.06 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$76.5 \mathrm{ms} \pm 584 \mathrm{μs}\left({\color{gray}4.44 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$85.2 \mathrm{ms} \pm 615 \mathrm{μs}\left({\color{gray}2.70 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$91.2 \mathrm{ms} \pm 604 \mathrm{μs}\left({\color{gray}1.93 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$43.8 \mathrm{ms} \pm 305 \mathrm{μs}\left({\color{gray}0.120 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$72.5 \mathrm{ms} \pm 478 \mathrm{μs}\left({\color{gray}-0.556 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$50.6 \mathrm{ms} \pm 376 \mathrm{μs}\left({\color{gray}1.05 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$60.0 \mathrm{ms} \pm 358 \mathrm{μs}\left({\color{gray}1.06 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$62.2 \mathrm{ms} \pm 388 \mathrm{μs}\left({\color{gray}0.883 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$62.1 \mathrm{ms} \pm 341 \mathrm{μs}\left({\color{gray}-0.134 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$126 \mathrm{ms} \pm 590 \mathrm{μs}\left({\color{gray}-1.620 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$135 \mathrm{ms} \pm 533 \mathrm{μs}\left({\color{gray}-2.366 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$19.3 \mathrm{ms} \pm 126 \mathrm{μs}\left({\color{gray}0.366 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$543 \mathrm{ms} \pm 1.03 \mathrm{ms}\left({\color{gray}2.99 \mathrm{\%}}\right) $$ Flame Graph

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

With execution.seedsPerTrial, one optimization.evaluate may legally run
up to 100 simulations sequentially, so the bindings validate the
seedsPerTrial reported by optimization.describe (integer, 1-100) and
multiply the per-response deadline by it. A new end-to-end test drives
a two-seed trial against the real built CLI, asserting the derived seed
sequence, per-seed replicates, and the mean objective.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

3 participants