fix(check): fail when the server withholds runtime rules for entitlement

.changeset/runtime-entitlement-withheld.md

@@ -0,0 +1,5 @@
+---
+"@taskless/cli": patch
+---
+
+`taskless check` now exits 1 when the Taskless service withholds a runtime rule because the organization's plan does not include runtime rules, instead of printing a notice and exiting 0. The output names the withheld rules and links to the upgrade page, and `--json` carries an `entitlement` object. If a CI job starts failing with this, the rules did not stop matching: they stopped running, and the fix is the plan, not the code. Unauthenticated and `--anonymous` runs, and `sg` and `vale` rules, are unchanged.

openspec/changes/archive/2026-09-27-runtime-entitlement-withheld/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-27

openspec/changes/archive/2026-09-27-runtime-entitlement-withheld/design.md

@@ -0,0 +1,86 @@
+## Context
+
+`planRuntime` (`rules/runtime/plan.ts`) turns a reconcile outcome into
+`{ execute, skipped, notices }`. `selectBlessedRuntimeRules` splits signed rules
+into `blessed` (signature in `run`) and `withheld` (everything else), and every
+`withheld` rule gets the same skip reason. `check` derives its exit code from
+`runEngines` alone, so no skip, of any kind, can fail a run.
+
+That was correct while every non-`run` outcome was either advisory (`unknown`,
+`missing`) or self-repairing (`unsafe`, restored for the next run). An
+entitlement withhold is neither: it is the server deliberately declining to run
+a rule the user believes is protecting them, and it will keep declining on every
+run until someone acts.
+
+## Decisions
+
+### 1. Withheld fails the run; the degrade paths still do not
+
+The standing spec says `check` SHALL NOT change the exit code because runtime
+rules were skipped on an **unverified** path (logged out, `--anonymous`,
+reconcile unreachable). Withheld is not that. Reconcile completed, the server
+answered, and the answer was "these will not run for you". Failing on it does
+not reopen the question of whether an outage should break CI.
+
+Exit code **1**, the code `check` already uses for error findings and engine
+failures. A distinct code was considered and rejected: every existing CI recipe
+treats non-zero as failure, the JSON envelope's `entitlement` field is the
+machine-readable discriminator, and a new code is a contract we would have to
+keep.
+
+`success` in `--json` is `false` whenever the exit code is non-zero, as today.
+
+### 2. Match withheld entries to local rules by reported path
+
+`entitlement.withheld` carries `{ ruleId, file }` and no signature, so the
+signature join `run` uses is not available. `file` is the path the CLI itself
+reported (the reported-path requirement already makes it contractual), so the
+join is exact: a signed rule whose reported `check.ts` path is in `withheld`
+is withheld for entitlement; every other non-`run` rule keeps today's reason.
+
+A `withheld` entry that matches no reported file is ignored for the skip list
+but still counts toward failing the run. The server said something will not
+run; the CLI not being able to name it locally is not a reason to go green.
+
+### 3. The entitlement object is parsed, not trusted
+
+`reconcile` normalizes `entitlement` into
+`{ runtimeSignatures: false; reason?; upgradeUrl?; withheld: [...] } | undefined`.
+It is `undefined` when the field is absent, not an object, or
+`runtimeSignatures` is not literally `false`. `withheld` entries without a string
+`file` are dropped. `upgradeUrl` is surfaced only when it parses as an absolute
+`https:` URL, so a malformed value is never printed as a link.
+
+Only `withheld` drives the exit code. `runtimeSignatures: false` with an empty
+`withheld` (an unentitled org reporting no matching runtime files) passes, since
+nothing the user holds was declined.
+
+### 4. One message, printed once
+
+Human output prints one notice per run naming the reason, the withheld rule
+names, and the upgrade URL, rather than repeating the URL per rule. Each rule's
+`skipped` entry gets the short reason `"not included in your Taskless plan"`
+so `--json` consumers reading `skipped` alone still see the right cause.
+
+### 5. Write-time warnings ride the existing notice channels
+
+`rule create` / `rule improve` already collect `notices` and print them as
+`Warning:` in human mode; restore notices already flow into `plan.notices`. The
+entitlement warning is one more entry in each, so no new output channel and no
+schema change beyond `check`'s `entitlement` field.
+
+The restore and retrieval responses are typed from the generated schema, which
+does not yet carry `entitlement`. They are widened with `MayCarryEntitlement<T>`
+(the field as optional `unknown`) and read through the same normalizer as
+reconcile, so the CLI ships assuming the field MIGHT be there instead of waiting
+on #207's deploy. Tightening that once the service always sends it is #409.
+
+## Risks
+
+- **A server bug that lists a file in `withheld` for an entitled org fails CI.**
+  Accepted: failing closed is the property runtime rules exist for, and the
+  failure names its cause and the URL to check.
+- **The vendored schema is stale in an unrelated way.** A refresh on 2026-09-27
+  also changes `successCases`/`failureCases` and drops `signature` from the
+  `sg`/`vale` file-set variants (the `runtime` variant keeps it). The cloud will
+  restore those signatures; regeneration waits for that (#409).

openspec/changes/archive/2026-09-27-runtime-entitlement-withheld/proposal.md

@@ -0,0 +1,98 @@
+## Why
+
+A runtime rule that stops running must never produce a green `check`. That is
+the whole reason runtime rules are gated on a server signature, and #403 is the
+one path where the CLI breaks it.
+
+The paid-cloud launch (taskless/taskless#207, `signature-entitlement`) stops
+blessing runtime rules for an organization whose plan lacks runtime signatures,
+for example after a paid plan lapses. The server keeps reconcile at **HTTP 200**
+and adds an additive `entitlement` object, so older CLIs keep working:
+
+```ts
+entitlement: {
+  runtimeSignatures: boolean;
+  reason?: "RUNTIME_SIGNATURES_NOT_IN_PLAN";
+  upgradeUrl?: string; // absolute, carries from=reconcile
+  withheld?: { ruleId: string; file: string }[];
+}
+```
+
+A withheld file appears in `withheld` and in none of `run`, `unsafe`,
+`unknown`, or `missing`. Restore and request retrieval carry the same object,
+without `withheld`, whenever they return a runtime file set.
+
+Every CLI through 0.11.2 reads only the four arrays. A withheld rule is absent
+from `run`, so it is skipped with the generic reason "not blessed by the server
+(unsafe / unknown / drift)", printed as a notice, and `check` exits 0. A
+customer whose plan lapses gets a green `taskless check` while their runtime
+rules have silently stopped running, and the one line that mentions it blames
+tampering that did not happen.
+
+The restore path has the mirror-image defect. Its completion notice promises
+that "the next `check` reports the repaired signature and is blessed through the
+ordinary path", which is false for an organization whose plan will not bless it.
+
+## What Changes
+
+- **`check` exits non-zero when `entitlement.withheld` is non-empty**, in human
+  and `--json` modes. Withheld is a verified outcome (reconcile completed and
+  answered), so it is not one of the degrade paths that deliberately never fail.
+- **`check` names the cause.** A withheld rule's skip reason says the plan does
+  not include runtime rules, not "unsafe / unknown / drift". The human output
+  prints `reason` and `upgradeUrl` once; `--json` carries an additive
+  `entitlement` object with `runtimeSignatures`, `reason`, `upgradeUrl`, and the
+  withheld rule names.
+- **The restore-completion notice stops promising blessing** when the restore
+  response carries `entitlement.runtimeSignatures: false`, and says instead that
+  the bytes were restored but will not run on this plan.
+- **`rule create` and `rule improve` warn when they write a runtime rule** from
+  a response carrying `entitlement.runtimeSignatures: false`: the rule is on
+  disk but will not run on this plan. The warning rides the existing `notices`
+  channel so `--json` callers see it too.
+- **The reconcile client parses `entitlement` defensively.** Absent, malformed,
+  or `runtimeSignatures: true` all mean "no entitlement outcome", so a server
+  that predates #207 changes nothing.
+
+Nothing changes for unauthenticated or `--anonymous` runs, for
+`--dangerously-run-scripts`, for the degrade paths (reconcile unreachable,
+401, no remote), or for `sg` and `vale` rules.
+
+## Capabilities
+
+### Modified Capabilities
+
+- `cli-check`: the exit-code requirement gains the withheld case; a new
+  requirement defines how a withheld rule is reported.
+- `cli-rule-reconciliation`: a new requirement defines how the CLI reads the
+  `entitlement` object and treats `withheld` as a disposition distinct from the
+  four buckets.
+- `cli-generated-rule-delivery`: a new requirement makes a runtime rule written
+  under a plan without runtime signatures say so, at write time and on restore.
+
+## Impact
+
+- `packages/cli/src/api/reconcile.ts`: parse `entitlement`.
+- `packages/cli/src/rules/runtime/plan.ts`, `run-set.ts`: classify withheld
+  rules, carry the entitlement onto the plan, fix the restore notice.
+- `packages/cli/src/commands/check.ts`, `schemas/check.ts`: exit code, human
+  output, `--json` field.
+- `packages/cli/src/commands/rules.ts`: write-time warning for create/improve.
+- `packages/cli/src/api/restore.ts`: surface `entitlement` from the restore body.
+- `packages/cli/src/agent/check.md`, `agent/ci.md`: document the new exit
+  condition, which a CI recipe must not read as a findings failure.
+- `api.schema.json` / `api.d.ts`: **not** regenerated here. #207 is not
+  deployed; the live `__schema` carries no `entitlement` today (measured
+  2026-09-27). Restore and retrieval type it as a field that might be present
+  (`MayCarryEntitlement<T>`); tightening once it always is, is #409.
+
+## Delivery shape
+
+**Single PR.** The change is one behavior (a withheld rule fails the run and
+says why) expressed at four call sites, plus the spec delta and tests, well
+inside the ~1200-line guidance. It is independently safe to ship before the
+server: with no `entitlement` in the response, every new branch is dead and
+behavior is byte-identical to today, which is also the property that lets it
+ship ahead of taskless/taskless#210 as the issue asks.
+
+Fixes #403

openspec/changes/archive/2026-09-27-runtime-entitlement-withheld/specs/cli-check/spec.md

@@ -0,0 +1,65 @@
+## MODIFIED Requirements
+
+### Requirement: Check subcommand exit codes reflect error severity
+
+The CLI SHALL exit with code 0 when no error-severity matches are found (including when only warnings, info, or hints exist) and no runtime rule was withheld for entitlement. The CLI SHALL exit with code 1 when at least one error-severity match is found. The CLI SHALL also exit with code 1 when reconciliation completed and the response's `entitlement.withheld` is non-empty, whatever the findings, in both human and `--json` modes.
+
+#### Scenario: Exit 0 when clean
+
+- **WHEN** the scanner produces zero results
+- **THEN** the process SHALL exit with code 0
+
+#### Scenario: Exit 0 when only warnings
+
+- **WHEN** the scanner produces results but none have severity "error"
+- **THEN** the process SHALL exit with code 0
+
+#### Scenario: Exit 1 when errors found
+
+- **WHEN** the scanner produces at least one result with severity "error"
+- **THEN** the process SHALL exit with code 1
+
+#### Scenario: Exit 1 when a runtime rule is withheld for entitlement
+
+- **WHEN** reconciliation returns a non-empty `entitlement.withheld` and the scan produces zero results
+- **THEN** the process SHALL exit with code 1
+- **AND** under `--json`, `success` SHALL be `false`
+
+#### Scenario: Entitlement without a withheld file does not fail
+
+- **WHEN** reconciliation returns `entitlement.runtimeSignatures: false` with an empty or absent `withheld`, and the scan produces zero results
+- **THEN** the process SHALL exit with code 0
+
+## ADDED Requirements
+
+### Requirement: Check reports runtime rules withheld for entitlement as a plan outcome
+
+When reconciliation completes and returns a non-empty `entitlement.withheld`, `taskless check` SHALL NOT execute any withheld rule, SHALL report each local runtime rule whose reported `check.ts` path appears in `withheld` as skipped with a reason stating that runtime rules are not included in the organization's plan, and SHALL NOT describe it as unsafe, unknown, drifted, or tampered. The human output SHALL include one notice naming the withheld rules, the `entitlement.reason`, and the `entitlement.upgradeUrl` when present. Under `--json`, the output SHALL carry an additive, optional `entitlement` object with `runtimeSignatures`, `reason`, `upgradeUrl`, and `withheld` (the local rule names), present only when reconciliation returned `runtimeSignatures: false`. This is a verified outcome and SHALL NOT be treated as one of the unverified paths that leave the exit code unchanged.
+
+#### Scenario: Withheld rule is named with its cause
+
+- **WHEN** an authenticated `check` reconciles and `entitlement.withheld` lists the reported `check.ts` of runtime rule `no-env-leak`
+- **THEN** `no-env-leak` SHALL NOT execute
+- **AND** its skip reason SHALL state that runtime rules are not included in the plan
+- **AND** its skip reason SHALL NOT mention unsafe, unknown, or drift
+
+#### Scenario: Upgrade URL is shown once
+
+- **WHEN** two runtime rules are withheld and `entitlement.upgradeUrl` is present
+- **THEN** the human output SHALL print the upgrade URL exactly once, in one notice naming both rules and the reason
+
+#### Scenario: Entitlement appears under --json
+
+- **WHEN** a runtime rule is withheld and `--json` is set
+- **THEN** stdout SHALL include `entitlement` with `runtimeSignatures: false`, `reason`, `upgradeUrl`, and `withheld` naming the rule
+- **AND** `skipped` SHALL still list the rule with its plan reason
+
+#### Scenario: A server without the entitlement object is unchanged
+
+- **WHEN** reconciliation returns the four arrays and no `entitlement` object
+- **THEN** `check` SHALL behave exactly as before this change, including its exit code and skip reasons
+
+#### Scenario: Degrade paths still never fail
+
+- **WHEN** `check` runs logged out, with `--anonymous`, or reconciliation cannot complete
+- **THEN** the exit code SHALL NOT change because runtime rules were skipped, as before this change

openspec/changes/archive/2026-09-27-runtime-entitlement-withheld/specs/cli-generated-rule-delivery/spec.md

@@ -0,0 +1,28 @@
+## ADDED Requirements
+
+### Requirement: A runtime rule written under a plan without runtime signatures says it will not run
+
+When the CLI writes a runtime rule from a response whose `entitlement.runtimeSignatures` is exactly `false` (from `rule create`, `rule improve`, or a restore during `check`), it SHALL still write the rule, and SHALL emit a warning that the rule is on disk but will not run on the organization's current plan, including the `upgradeUrl` when it is an absolute `https:` URL. The warning SHALL be carried in the command's existing notices, so it appears in human output and under `--json`. A restore under such a response SHALL NOT state or imply that the next `check` will bless or run the rule. A response with no `entitlement` object, or with `runtimeSignatures: true`, SHALL produce no such warning.
+
+#### Scenario: Created runtime rule under an unentitled plan
+
+- **WHEN** `rule create` receives a generated runtime rule with `entitlement.runtimeSignatures: false`
+- **THEN** the CLI SHALL write the rule
+- **AND** SHALL warn that it will not run on the current plan
+- **AND** under `--json` the warning SHALL appear in `notices`
+
+#### Scenario: Restore under an unentitled plan does not promise blessing
+
+- **WHEN** a restore during `check` writes a runtime rule and the restore response carries `entitlement.runtimeSignatures: false`
+- **THEN** the restore notice SHALL say the rule's bytes were restored but will not run on the current plan
+- **AND** SHALL NOT say the next `check` blesses it
+
+#### Scenario: Entitled or legacy responses do not warn
+
+- **WHEN** a runtime rule is written from a response with no `entitlement` object or with `runtimeSignatures: true`
+- **THEN** the CLI SHALL emit no entitlement warning
+
+#### Scenario: Static rules never warn
+
+- **WHEN** `rule create` writes an `sg` or `vale` rule
+- **THEN** the CLI SHALL emit no entitlement warning

openspec/changes/archive/2026-09-27-runtime-entitlement-withheld/specs/cli-rule-reconciliation/spec.md

@@ -0,0 +1,32 @@
+## ADDED Requirements
+
+### Requirement: The CLI reads the reconcile entitlement object
+
+The CLI SHALL read the optional `entitlement` object on a successful reconcile response. It SHALL treat the organization as unentitled only when `entitlement.runtimeSignatures` is exactly `false`; an absent, malformed, or `true` value SHALL be treated as no entitlement outcome, leaving the four buckets to drive execution exactly as before. For an unentitled response the CLI SHALL read `reason`, `upgradeUrl`, and `withheld` (a list of `{ ruleId, file }`), SHALL drop a `withheld` entry without a string `file`, and SHALL surface `upgradeUrl` only when it is an absolute `https:` URL.
+
+`entitlement.withheld` SHALL be a disposition distinct from the four buckets: a withheld file SHALL NOT be executed, SHALL NOT be surfaced as tamper, drift, or never-issued, and SHALL NOT be sent to restore. The CLI SHALL match a withheld entry to a local runtime rule by the `file` it reported, since the entry carries no signature. A withheld entry that matches no reported file SHALL still count as withheld for the purpose of the exit code.
+
+#### Scenario: Absent entitlement changes nothing
+
+- **WHEN** a reconcile response carries no `entitlement` object
+- **THEN** the CLI SHALL drive execution from `run`, `unsafe`, `unknown`, and `missing` exactly as before
+
+#### Scenario: Entitled response changes nothing
+
+- **WHEN** a reconcile response carries `entitlement: { runtimeSignatures: true }`
+- **THEN** the CLI SHALL drive execution from the four buckets exactly as before
+
+#### Scenario: Withheld is matched by reported path
+
+- **WHEN** `entitlement.withheld` lists `{ ruleId, file }` and `file` equals the path the CLI reported for a runtime rule's `check.ts`
+- **THEN** the CLI SHALL classify that rule as withheld for entitlement and SHALL NOT execute it
+
+#### Scenario: Withheld is not repaired
+
+- **WHEN** a runtime rule is withheld for entitlement
+- **THEN** the CLI SHALL NOT request a restore for it
+
+#### Scenario: A malformed upgrade URL is not shown
+
+- **WHEN** `entitlement.upgradeUrl` is not an absolute `https:` URL
+- **THEN** the CLI SHALL omit it from human and `--json` output