Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,7 @@ write boundary:
| --- | --- | --- |
| Turn a public issue into a reviewable, evidence-backed change | [Issue Fix](loopx/capabilities/issue_fix/README.md) | `loopx capability show issue-fix --format json` |
| Qualify the exact final diff before delivery | [Change Quality](loopx/capabilities/change_quality/README.md) | `loopx capability show change-quality-qualification --format json` |
| Notice busy-but-off-goal work rounds before the periodic review | [Progress-Review Sentinel](loopx/capabilities/progress_review/README.md) | `loopx capability show progress-review-sentinel --format json` |
| Preserve a changing stack of already reviewed branches | [Integration Branch](loopx/capabilities/integration_branch/README.md) | `loopx capability show integration-branch-reconcile --format json` |
| Explore uncertain research without losing hypotheses and findings | [Explore](loopx/capabilities/explore/README.md) | `loopx capability show explore --format json` |
| Rebase decisions on current evidence and verified outcomes | [Decision Context](loopx/capabilities/decision_context/README.md) | `loopx capability show decision-context --format json` |
Expand Down
1 change: 1 addition & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,7 @@ Capability 把上述通用原语组成 outcome-owned 工作泳道。先按结果
| --- | --- | --- |
| 把公开 issue 推进为可审查、有证据的变更 | [Issue Fix](loopx/capabilities/issue_fix/README.zh-CN.md) | `loopx capability show issue-fix --format json` |
| 在交付前对精确 final diff 做质量验收 | [Change Quality](loopx/capabilities/change_quality/README.md) | `loopx capability show change-quality-qualification --format json` |
| 在周期复审之前发现“忙碌但偏离目标”的工作轮次 | [进展评估哨兵](loopx/capabilities/progress_review/README.zh-CN.md) | `loopx capability show progress-review-sentinel --format json` |
| 维护由多个已审查分支组成、持续变化的集成栈 | [Integration Branch](loopx/capabilities/integration_branch/README.md) | `loopx capability show integration-branch-reconcile --format json` |
| 在不丢失假设和发现的前提下探索不确定研究问题 | [Explore](loopx/capabilities/explore/README.zh-CN.md) | `loopx capability show explore --format json` |
| 基于当前证据和已验证结果重新建立决策上下文 | [Decision Context](loopx/capabilities/decision_context/README.zh-CN.md) | `loopx capability show decision-context --format json` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ const capabilityCopy: Record<WorkspaceLocale, Record<string, LocalizedCopy>> = {
displayName: "Change quality qualification",
description: "Prepares a provider-neutral review packet, allows at most one policy-authorized safe-fix pass, and can require an exact-diff receipt.",
},
progress_review: {
displayName: "Progress-review sentinel",
description: "Records typed drift receipts from an external bounded review of scoped file deltas; assist may raise the existing autonomous replan obligation.",
},
explore_graph: {
displayName: "Explore Graph",
description: "Organizes bounded exploration as a typed evidence graph so branches, findings, and synthesis remain inspectable.",
Expand Down Expand Up @@ -85,6 +89,10 @@ const capabilityCopy: Record<WorkspaceLocale, Record<string, LocalizedCopy>> = {
displayName: "变更质量验证",
description: "生成与 Provider 无关的审阅包,最多允许一次策略授权的安全修复,并可要求精确 diff 回执。",
},
progress_review: {
displayName: "进展评估哨兵",
description: "记录外部有界评估对限定文件变化给出的类型化漂移回执;assist 模式可触发已有的自主重规划义务。",
},
explore_graph: {
displayName: "探索图谱",
description: "把有界探索组织为 typed 证据图,让探索分支、发现与综合结论都可检查、可追溯。",
Expand Down
14 changes: 13 additions & 1 deletion docs/architecture/rfcs/optional-semantic-assistance-jev-v0.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# RFC: Agent Judgment and Optional Independent Assessment — Jev as a Candidate (v0)

- **RFC status:** Draft; M0 **accepted-for-discussion** ([maintainer decision](https://github.com/loopx-project/loopx/pull/4749#pullrequestreview-5259253204)). Q1–Q7 remain pending; the research/design is not accepted for implementation.
- **Delivery maturity:** Proposal; documentation only, no integration or model qualification.
- **Delivery maturity:** Research proposal; a separate D1-only optional shadow implementation is proposed in Appendix A. No model qualification or automatic correction is established. The default-off sentinel capability and its recorded differential live in [`loopx/capabilities/progress_review`](../../../loopx/capabilities/progress_review/README.md); those numbers do not change this Draft's status.
- **Created:** 2026-09-19. **Last normative revision:** 2026-09-20.
- **Implementation baseline:** `9f1916960306b3650d795895b89f331eeae2516e`; source ownership and trigger behavior rechecked at PR revision `27812bd0fb437f831a541b564bcb5be8a96ff77e`. Historical upstream inspection is recorded in Appendix A, not a whole-system certification.
- **Authors / owners:** Proposal author; existing domain maintainers own any direction selected. No new runtime authority or assigned implementation owner.
Expand Down Expand Up @@ -355,8 +355,19 @@ No default D1 implementation, automatic worker adoption or hidden mandatory-mode
- **Delta:** D7/D8, provisional opportunity ordering, expected-value decomposition and bounded ranking comparisons; bilingual text and index updated.
- **Evidence/remaining gap:** source inspection and documentation only. No live provider comparison, production ranker or new authority; Q1–Q7 remain pending.

### 2026-09-21 — Task-progress shadow implementation proposal (RFC D1)

- **Baseline:** upstream `62d18677c`; the implementation includes only the optional D1 command, without D2–D8 ranking or selector changes.
- **Proposal:** scoped checkpoint capture around actual refresh-state, separate inference, off/shadow configuration and historical readback. See the [operation guide](../../../packages/loopx-jev/DRIFT_SHADOW.md).
- **Evidence boundary:** [discussion evolution and current-implementation measurements](../../../packages/loopx-jev/DESIGN_DECISIONS.md); deterministic integration checks are distinct from provider accuracy and task benefit. No automatic intervention or native host-hook rollout.
- **Pending:** maintainer acceptance of this optional-tool scope and independently evaluated comparative value. The earlier M0 intake is not retroactive implementation approval.

## Appendix B: Decision log

The separately proposed [Task-progress observation tool and decision record](../../../packages/loopx-jev/DESIGN_DECISIONS.md)
do not change the historical M0 decision or settle Q1–Q7. Maintainers review that
optional-tool scope separately; experimental results do not establish product adoption.

| Date | Proposal / decision | Owner / approval state | Alternatives | Sections |
| --- | --- | --- | --- | --- |
| 2026-09-19 | D1 CLI, optional Jev package and profile v1 recommended | Not approved; superseded as the default recommendation by this revision | D2 or separate configuration | 3, 5, 11, 12 |
Expand All @@ -375,6 +386,7 @@ Record any future accepting decision with its actual public link and exact scope
| E3 | TypeSafe API/confidence/model-limitations pages linked in section 5.3 | Interface and documented limitations; not independent model quality, data terms or a live trial |
| E4 | PR #4749 and its linked maintainer review | Public request and request-changes rationale; no accepted research/adoption decision |
| E5 | A/B/C and F01–F12 | Proposed experiments/obligations; unexecuted for this feature |
| E6 | [Task-progress observation decision record](../../../packages/loopx-jev/DESIGN_DECISIONS.md) | Public synthesis and current-implementation observations; not independent qualification, complete A/B/C or automatic-correction evidence |

## Appendix D: Deferred mechanisms and rejected shortcuts

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# RFC:Agent 判断与可选独立评估——以 Jev 为候选方案(v0)

- **RFC status:** Draft;M0 **accepted-for-discussion(接受为讨论稿)**([维护者决定](https://github.com/loopx-project/loopx/pull/4749#pullrequestreview-5259253204))。Q1–Q7 仍待决;研究/设计未获实施批准。
- **Delivery maturity:** Proposal;仅文档,没有接入实现或模型质量验收
- **Delivery maturity:** 研究提案;附录 A 单独提出仅 D1 的可选 shadow 实现,没有建立模型质量资格或自动纠正效果。默认关闭的哨兵 capability 及其录制对照见 [`loopx/capabilities/progress_review`](../../../loopx/capabilities/progress_review/README.zh-CN.md);这些数字不改变本 Draft 的状态
- **Created:** 2026-09-19。**Last normative revision:** 2026-09-20。
- **Implementation baseline:** `9f1916960306b3650d795895b89f331eeae2516e`;在 PR 版本 `27812bd0fb437f831a541b564bcb5be8a96ff77e` 重新核对源码归属与触发器行为。历史 upstream 检查记于附录 A,不构成全系统认证。
- **Authors / owners:** 提案作者;被选方向由现有领域维护者负责。不新增运行时权威,也未指派实施 owner。
Expand Down Expand Up @@ -355,8 +355,18 @@ M0 不默认批准 D1 实施、自动 worker 采纳或隐藏的必需模型阶
- **增量:** D7/D8、暂定机会顺序、预期价值分解与有界排序比较;同步双语正文及索引。
- **证据/剩余缺口:** 仅源码检查和文档。没有 live 提供方比较、生产 ranker 或新权限;Q1–Q7 仍待决。

### 2026-09-21 — 任务进展旁路观察实现提案(RFC D1)

- **基线:** upstream `62d18677c`;当前实现只包含可选 D1 命令,不引入 D2–D8 排序或选择器改动。
- **提案:** 真实 refresh-state 前后限定检查点采集、独立推理、off/shadow 配置和历史读回。参见[操作指南](../../../packages/loopx-jev/DRIFT_SHADOW.zh-CN.md)。
- **证据边界:** [讨论演进与当前实现的运行结果](../../../packages/loopx-jev/DESIGN_DECISIONS.zh-CN.md);确定性集成检查不同于模型准确率或任务收益。没有自动干预或原生宿主 hook 推广。
- **待决:** 维护者是否接受这个可选工具范围,以及独立评估的比较价值。不能将原 M0 收录倒推为实现已获批准。

## 附录 B:决策日志

单独提出的 [任务进展旁路观察工具及决策记录](../../../packages/loopx-jev/DESIGN_DECISIONS.zh-CN.md)
不改变历史 M0 决定,也不替代 Q1–Q7。维护者单独评审可选工具范围,实验结果不等于产品采用。

| 日期 | 提案 / 决定 | owner / 批准状态 | 替代方案 | 章节 |
| --- | --- | --- | --- | --- |
| 2026-09-19 | 曾推荐 D1 CLI、可选 Jev 包和 profile v1 | 未批准;本修订撤下其默认推荐地位 | D2 或独立配置 | 3、5、11、12 |
Expand All @@ -375,6 +385,7 @@ M0 不默认批准 D1 实施、自动 worker 采纳或隐藏的必需模型阶
| E3 | 第 5.3 节 TypeSafe API/confidence/模型限制页面 | 接口与公开限制,不是独立模型质量、数据条款或 live 实验 |
| E4 | PR #4749 及链接的维护者评审 | 公开请求和请求修改理由,不是研究/采用已获接受 |
| E5 | A/B/C 与 F01–F12 | 拟议实验/义务,该功能尚未执行 |
| E6 | [任务进展观察决策记录](../../../packages/loopx-jev/DESIGN_DECISIONS.zh-CN.md) | 公开论证摘要和当前实现的观察,不是独立资格、完整 A/B/C 或自动纠正证据 |

## 附录 D:延后机制与排除的捷径

Expand Down
92 changes: 92 additions & 0 deletions examples/progress-review-sentinel-smoke.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""Smoke-test the progress-review sentinel differential from recorded answers.

Replays the committed comparison matrix against the committed provider
recordings, so it needs no credential and no network. It asserts the harness
shape and that the replay reproduces the committed summary: the typed repeat
fuse stays quiet on every self-declared `advanced` round, while each receipt
signal's first-flag rounds match what the live run recorded.
"""

from __future__ import annotations

import json
from pathlib import Path
import sys
import tempfile

REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO_ROOT))
sys.path.insert(1, str(REPO_ROOT / "packages" / "loopx-jev" / "src"))

from loopx_jev.sentinel_compare import COMPARISON_SCHEMA, compare # noqa: E402
from loopx_jev.sentinel_matrix import load_sentinel_matrix # noqa: E402

FIXTURES = REPO_ROOT / "packages" / "loopx-jev" / "tests" / "fixtures" / "sentinel"


def deterministic_view(comparison: dict) -> dict:
"""Project the fields that must reproduce from recordings alone."""

return {
case["case_id"]: {
"first_flag_round": case["first_flag_round"],
"first_obligation_round": case["first_obligation_round"],
"typed_repeat_first_round": case["baseline"]["typed_repeat_first_round"],
"statuses": [row["status"] for row in case["rounds"]],
}
for case in comparison["cases"]
}


def main() -> int:
matrix = load_sentinel_matrix(FIXTURES / "matrix.json")
expected_path = FIXTURES / "expected_summary.json"
if not expected_path.is_file():
print("expected_summary.json is missing; record it with `loopx-jev sentinel compare --live`")
return 1
expected = json.loads(expected_path.read_text(encoding="utf-8"))
with tempfile.TemporaryDirectory(prefix="loopx-sentinel-smoke-") as temporary:
comparison = compare(
matrix,
responses=FIXTURES / "responses",
live=False,
model=expected["model"],
deadline_ms=5000,
drift_threshold=2,
)
Path(temporary, "comparison.json").write_text(json.dumps(comparison), encoding="utf-8")
assert comparison["schema_version"] == COMPARISON_SCHEMA
assert comparison["execution"] == "recorded_replay"
assert comparison["aggregate"]["cases"] == 16
assert comparison["aggregate"]["baseline"]["typed_repeat_fired_cases"] == 0
for case in comparison["cases"]:
assert all(row["status"] != "not_captured" for row in case["rounds"]), case["case_id"]
assert all(
row["execution_kind"] == "recorded_replay"
for row in case["rounds"]
if row["status"] in {"completed", "abstained", "failed"}
), case["case_id"]
actual = deterministic_view(comparison)
if actual != expected["deterministic_view"]:
for case_id, view in actual.items():
if view != expected["deterministic_view"].get(case_id):
print("mismatch", case_id, json.dumps(view), json.dumps(expected["deterministic_view"].get(case_id)))
return 1
print(
json.dumps(
{
"status": "ok",
"cases": comparison["aggregate"]["cases"],
"signals": comparison["aggregate"]["signals"],
"baseline": comparison["aggregate"]["baseline"]["typed_repeat_fired_cases"],
},
indent=2,
sort_keys=True,
)
)
return 0


if __name__ == "__main__":
raise SystemExit(main())
2 changes: 2 additions & 0 deletions loopx/capabilities/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
from .connector_registry.catalog_entry import CONNECTOR_REGISTRY_CATALOG_ENTRY
from .external_research.catalog_entry import EXTERNAL_RESEARCH_CATALOG_ENTRY
from .reliability_diagnostics.catalog_entry import RELIABILITY_DIAGNOSTICS_CATALOG_ENTRY
from .progress_review.catalog_entry import PROGRESS_REVIEW_CATALOG_ENTRY
from .registry import CapabilityRegistry

CAPABILITY_CATALOG_SCHEMA_VERSION = "loopx_capability_catalog_v0"
Expand Down Expand Up @@ -56,6 +57,7 @@
CONNECTOR_REGISTRY_CATALOG_ENTRY,
EXTERNAL_RESEARCH_CATALOG_ENTRY,
RELIABILITY_DIAGNOSTICS_CATALOG_ENTRY,
PROGRESS_REVIEW_CATALOG_ENTRY,
)
# Preserve the original import surface while routing all reads through the registry.
CAPABILITIES = BUILTIN_CAPABILITIES
Expand Down
42 changes: 42 additions & 0 deletions loopx/capabilities/configuration_ui.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@

from ..configuration_transaction import configuration_payload_revision

from .progress_review.policy import (
PROGRESS_REVIEW_MAX_DRIFT_THRESHOLD,
PROGRESS_REVIEW_MIN_DRIFT_THRESHOLD,
PROGRESS_REVIEW_MODES,
PROGRESS_REVIEW_SIGNALS,
)

CAPABILITY_CONFIGURATION_CATALOG_SCHEMA = "capability_configuration_catalog_v0"
CAPABILITY_CONFIGURATION_EDITOR_SCHEMA = "capability_configuration_editor_v0"
CAPABILITY_CONFIGURATION_RESOLUTION_SCHEMA = "capability_configuration_resolution_v0"
Expand Down Expand Up @@ -288,6 +295,41 @@ def capability_configuration_editor(
_field("strict_receipt", "Require an exact-diff receipt", "boolean"),
],
},
"progress_review": {
"supported_scopes": ["goal"],
"writable_scopes": ["goal"],
"fields": [
_field(
"mode",
"Mode",
"select",
options=PROGRESS_REVIEW_MODES,
required=True,
description=(
"off records nothing; shadow records typed receipts only; "
"assist lets consecutive drift receipts raise the existing "
"autonomous replan obligation. No pause or gate authority."
),
),
_field(
"signal",
"Drift signal",
"select",
options=PROGRESS_REVIEW_SIGNALS,
description=(
"Which receipt judgment counts as drift: the Noul behavior/"
"acceptance pair or the Choice relation/increment pair."
),
),
_field(
"drift_threshold",
"Consecutive drift receipts before an obligation",
"integer",
minimum=PROGRESS_REVIEW_MIN_DRIFT_THRESHOLD,
maximum=PROGRESS_REVIEW_MAX_DRIFT_THRESHOLD,
),
],
},
"pull_request_review": {
"supported_scopes": ["machine", "goal"],
"writable_scopes": ["machine", "goal"],
Expand Down
Loading
Loading