Skip to content

Loss decoding based on Pauli envelope - #166

Open
Yue Wu (yuewuo) wants to merge 165 commits into
mainfrom
feature-90-pauli-envelop-loss
Open

Loss decoding based on Pauli envelope#166
Yue Wu (yuewuo) wants to merge 165 commits into
mainfrom
feature-90-pauli-envelop-loss

Conversation

@yuewuo

@yuewuo Yue Wu (yuewuo) commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Adds end-to-end Pauli-envelope loss decoding support:

  • Compiles configurable gate-loss policies into causal loss graphs while preserving herald identity across gadgets and decode windows.
  • Adds language support for the Pauli-envelope data that can is output by the annotator and can be inspected or edited by user
  • Adds coordinator strategies to ignore loss, convert it into shot-scoped edge reweights, or pass structured LossInfo to loss-aware decoders.
  • Supports decoder capability negotiation, persistent graph reuse, and random imputation of lost measurement outcomes.
  • Includes a herald-aware MLE reference decoder, updated protobuf/Python/CLI APIs, comprehensive tests, and tutorial documentation.

Yue Wu and others added 30 commits June 18, 2026 10:36
Merges 8 commits from main including PR #84 (@REPROPAGATE).

Conflict resolution:
- compose_builder.py: kept HEAD (local @REPROPAGATE implementation
  is a superset of main's; includes _translate_compose_conditionals).
- jit_annotate.py: kept HEAD (CONDITIONAL R<j> emission block).
- jit_noise_builder.py: merged error message wording (main's 'or' +
  HEAD's extra_clause).
- teleportation.deq fixture: kept HEAD (additional Teleporatation2
  test compose).
- test_compose_repropagate.py: kept HEAD (added
  TestTranslateComposeConditionals class).
- jit_library_builder_test.py: kept both (HEAD's 8 compose_conditional
  tests + main's 3 build_jit_program tests are non-overlapping).

Known integration issues (to fix in follow-up commits):
- 9 tests fail due to divergent @REPROPAGATE implementations between
  HEAD's f526417 and main's PR #84.  The basis-freedom check in
  jit_noise_builder.py:1788 doesn't see row_extra_descs for CONDITIONAL
  contributions in the merged code.
- 2 runtime test files fail to collect (deq_runtime needs maturin rebuild
  for main's new APIs).
Co-authored-by: Yue Wu <yuewu4@microsoft.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…t, fix swapped Clifford gates (#100)

Co-authored-by: jbellorivas <jbellorivas@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fn edge(&self, index: usize) -> (f64, &'a [u64]) {
let start = usize::try_from(self.edge_offsets[index]).expect("CSR offset exceeds usize");
let end = usize::try_from(self.edge_offsets[index + 1]).expect("CSR offset exceeds usize");
let start = usize::try_from(self.edge_offsets[index]).unwrap();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why switch from expect to unwrap?

Comment thread deq/deq/cli/annotate.py
#: suppress the interactive Mako safety prompt
skip_mako_warning: bool = False,
#: physical loss model: "neutral-atom", "trapped-ion", "none", or a .py file
loss_model: str = "neutral-atom",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Literal['neutral-atom', 'trapped-ion', 'none'] as type instead of str. Consider an enum if it makes sense.

Comment thread deq/deq/cli/jit.py
#: defaults to: (logical CPU count - 2), minimum 1
jobs: int = max((os.cpu_count() or 1) - 2, 1),
#: physical loss model: "neutral-atom", "trapped-ion", "none", or a .py file
loss_model: str = "neutral-atom",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Literal[...] or some enum class.

Comment thread deq/deq/cli/simulate.py
debug_dir: str | None,
simulator: str = "static",
loss_config: dict[str, object] | None = None,
timeout: float = 36000,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These hard-coded defaults ought to be dealt with. The simplest way would be to centralize them somewhere (e.g., a defaults.py module with documented constants with names such as DEFAULT_TIMEOUT).

Comment thread deq/deq/cli/simulate.py
cmd += ["--coordinator-config", coordinator_config]

runtime_env = os.environ.copy()
runtime_env.setdefault("TOKIO_WORKER_THREADS", "4")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing with these string literal constants: let's do something about them.

<span class="line"><span style="color:#0000FF"> ERROR</span><span style="color:#000000">(</span><span style="color:#098658">0.01</span><span style="color:#000000">) </span><span style="color:#267F99">C0</span><span style="color:#800000"> OUT0.LX0</span></span>
<span class="line"><span style="color:#0000FF"> ERROR</span><span style="color:#000000">(</span><span style="color:#098658">0.01</span><span style="color:#000000">) </span><span style="color:#267F99">C0</span><span style="color:#267F99"> C1</span><span style="color:#800000"> OUT0.LX0</span></span>
<span class="line"><span style="color:#0000FF"> ERROR</span><span style="color:#000000">(</span><span style="color:#098658">0.01</span><span style="color:#000000">) </span><span style="color:#267F99">C1</span><span style="color:#800000"> OUT0.LX0</span></span>
<span class="line"><span style="color:#795E26"> @SIMULATE_ONLY</span></span>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The embedded HTML tags read poorly outside of a web browser. And even in a web browser they could be simplified with CSS.

Comment on lines +256 to +263
for (index, site) in loss.sites.iter().enumerate() {
for &edge in &site.source_edges {
activate(edge, site.probability, &mut order);
}
for &edge in &site.continuation_edges {
activate(edge, accumulated[index], &mut order);
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same error mechanism (edge) can appear in source_edges and continuation_edges for one loss site. Processing those entries independently double-counts the physical loss and artificially increases the edge's activation and reweighted probability.

Comment on lines +120 to +128
for edge in range(num_edges):
if edge in loss_edges:
continue # free envelope edge, bounds [0, 1], weight 0
weight = self.weights[edge]
if math.isinf(weight):
if weight > 0.0:
upper[edge] = 0.0 # p <= 0 non-loss edge: unusable
else:
lower[edge] = 1.0 # p >= 1: certain error

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transpiler can map an ordinary Pauli error and a loss-envelope generator to the same decoder edge when their detectors match. The MLE decoder then treats the shared edge as loss-only, which can make an otherwise valid decoding problem infeasible. See:

from importlib.util import module_from_spec, spec_from_file_location
from pathlib import Path
from types import SimpleNamespace as Namespace


def check_mixed_regular_loss_edge() -> None:
    module_path = Path("deq/deq_runtime/src/decoder/mle_loss_decoder.py")
    spec = spec_from_file_location("mle_loss_decoder", module_path)
    assert spec is not None and spec.loader is not None
    module = module_from_spec(spec)
    spec.loader.exec_module(module)

    graph = Namespace(
        vertex_num=2,
        hyperedges=[
            Namespace(vertices=[0], probability=0.001),
            Namespace(vertices=[1], probability=0.0),
        ],
    )

    def site(*, source=(), children=()):
        return Namespace(
            source_edges=list(source),
            continuation_edges=[],
            children=list(children),
            heralds=[0],
            probability=0.1,
        )

    def decode(sites):
        return module.Decoder(graph).decode(
            [0, 1],
            Namespace(sites=sites),
        )

    try:
        decode([
            site(source=(0,), children=(1,)),
            site(source=(1,)),
        ])
    except RuntimeError as error:
        assert "infeasible" in str(error).lower()
        print(f"mixed case: {error}")
    else:
        raise AssertionError(
            "mixed regular/loss edge unexpectedly remained feasible"
        )

    regular_control = decode([
        site(children=(1,)),
        site(source=(1,)),
    ])
    conflict_control = decode([
        site(source=(0,)),
        site(source=(1,)),
    ])

    assert regular_control == [0, 1]
    assert conflict_control == [0, 1]
    print(f"regular control: {regular_control}")
    print(f"conflict control: {conflict_control}")


check_mixed_regular_loss_edge()

Comment on lines +87 to +92
def decode(self, syndrome, loss=None) -> list[int]:
num_edges = self.num_edges
sites = list(loss.sites) if loss is not None else []
self._validate_loss_sites(sites)
if num_edges == 0:
return []

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corner case: this returns an empty correction without checking whether the syndrome is empty. A graph with one detector, no edges, and syndrome {0} is accepted despite the fact that it can't be explained.

Comment on lines +223 to +238
options = {}
if self.time_limit is not None:
options["time_limit"] = float(self.time_limit)

result = milp(
c=objective,
constraints=constraints,
integrality=np.ones(num_vars, dtype=int),
bounds=Bounds(lower, upper),
options=options,
)
if result.x is None:
raise RuntimeError(
f"loss decoder MILP produced no solution (status={result.status}): "
f"{result.message}"
)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This RuntimeError will trigger a panic instead of returning a nicer, more intelligible decoder failure.

@jmbr

Copy link
Copy Markdown
Member

deq's transpile --detectors raises an exception when the generated Stim circuit contains LOSS_ERROR. The circuit is written to disk without DETECTOR or OBSERVABLE_INCLUDE annotations.

To reproduce:

deq transpile \
  deq/documents/tutorial/examples/loss-simulation/repetition_code.deq \
  --out /tmp/repetition-loss.deq.jit \
  --program Memory \
  --jobs 1 \
  --mako d=3 \
  --mako p=0.001 \
  --mako p_loss=0.01 \
  --mako rounds=9 \
  --mako replenish=0 \
  --skip-mako-warning \
  --detectors

The output:

Generated JIT library: /tmp/repetition-loss.deq.jit
  Port types: 1
  Gadget types: 3
  Program instructions: 11
Generated stim circuit: /tmp/repetition-loss.stim
Traceback (most recent call last):
  File "/home/jmbr/sources/qdk-ec-pr166/.venv/bin/deq", line 10, in <module>
    sys.exit(run())
             ^^^^^
  File "/home/jmbr/sources/qdk-ec-pr166/deq/deq/cli/__init__.py", line 45, in run
    arguably.run()
  File "/home/jmbr/sources/qdk-ec-pr166/.venv/lib/python3.12/site-packages/arguably/_context.py", line 732, in run
    result = cmd.call(parsed_args)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmbr/sources/qdk-ec-pr166/.venv/lib/python3.12/site-packages/arguably/_commands.py", line 409, in call
    return self.function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmbr/sources/qdk-ec-pr166/deq/deq/cli/jit.py", line 113, in transpile
    _annotate_stim_with_detectors(jit_library, _stim_path_for(out), assertions)
  File "/home/jmbr/sources/qdk-ec-pr166/deq/deq/cli/jit.py", line 310, in _annotate_stim_with_detectors
    body = stim.Circuit.from_file(stim_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Gate not found: 'LOSS_ERROR'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants