From 90c84ef11acd00b5581f850a40aa9ff04e9b4bcb Mon Sep 17 00:00:00 2001 From: abrichr Date: Sun, 26 Jul 2026 19:49:33 -0400 Subject: [PATCH] docs: add qualification cockpit journey --- docs/guides/index.md | 5 ++ docs/guides/qualify-a-workflow.md | 133 ++++++++++++++++++++++++++++++ docs/reference/cli.md | 53 ++++++++++++ mkdocs.yml | 1 + 4 files changed, 192 insertions(+) create mode 100644 docs/guides/qualify-a-workflow.md diff --git a/docs/guides/index.md b/docs/guides/index.md index 1cc2dfe..f79f54e 100644 --- a/docs/guides/index.md +++ b/docs/guides/index.md @@ -23,6 +23,11 @@ How-to guides for real deployments. Each assumes you have Use `lint` and `certify` to make "runnable" distinct from "safe." +- [__Qualify a workflow__](qualify-a-workflow.md) + + Review the graph and risk, bind identity and effects, run representative + and fault cases, certify, seal, export, and deploy from Desktop or CLI. + - [__Run a deployment__](run-a-deployment.md) Wire a real backend, effect verification, actuation, and durability from diff --git a/docs/guides/qualify-a-workflow.md b/docs/guides/qualify-a-workflow.md new file mode 100644 index 0000000..1bccb31 --- /dev/null +++ b/docs/guides/qualify-a-workflow.md @@ -0,0 +1,133 @@ +# Qualify a workflow + +Qualification turns a compiled demonstration into a governed artifact tied to +one application, version, execution surface, and environment. It is the review +between **compiled** and **ready to deploy**. + +OpenAdapt Desktop provides the complete guided path. The same contract is +available through `openadapt flow qualify` for automation and integration. + +## Open the qualification cockpit + +In Desktop, open a compiled workflow and choose **Qualification**. For a new +project, define: + +- the execution surface: Browser, Windows, macOS, Linux, RDP, or Citrix; +- the application and version; +- a stable environment identifier shared with the qualification runner; +- any required runner capabilities; and +- the minimum acceptable effect-verification tier. + +Starting the project retains the compiled workflow and creates a versioned +qualification contract. Later edits advance its revision and invalidate stale +case evidence and certification. + +## Review the compiled graph and every action + +The **Workflow structure** view shows actions, branches, loops, exception paths, +terminal outcomes, and the retained targeting ladder. Review the inferred +program before assigning safety contracts. + +For every action, confirm one business-risk class: + +- **Read-only** observes without changing business state. +- **State-changing** changes UI or transient application state. +- **Consequential** can change a meaningful business record or trigger an + external effect. +- **Irreversible** is consequential and cannot be safely undone through the + qualified workflow. + +The cockpit explains unresolved classifications and coverage gaps under **What +must be resolved**. Certification refuses an unreviewed action rather than +guessing. + +## Arm identity before consequential actions + +Select a consequential action under **Arm identity and bind the business +effect**. Identity can use the canonical evidence ladder or a quorum of explicit +signals such as application, session, workflow state, subject name, record ID, +and secondary identifier. + +Each signal declares its source, exact or normalized comparison, and—when +needed—its screen region, parameter binding, or extraction rule. Duplicate or +conflicting records remain refusal conditions. + +## Bind the resulting business effect + +Identity proves OpenAdapt is acting on the intended record. Effect verification +proves the intended state persisted. Bind each consequential write to a verifier +and choose the minimum accepted strength: + +1. **Independent system**: API, database, audit feed, or downstream record. +2. **Independent session**: a separate read-only session retrieves the result. +3. **Persisted-state reacquisition**: leave the record, find it again, and read + the saved value. +4. **Immediate screen**: the current field, banner, or notification. + +The workflow-level minimum and each effect binding are both enforced. A weaker +observation cannot be promoted to `VERIFIED` merely because the UI looked +successful. + +## Run representative and fault cases + +Add the normal cases the workflow must complete and the uncertainty cases it +must refuse. The cockpit supports representative, ambiguity, wrong-identity, +stale-identity, weak-effect, and missing-effect cases. + +Choose the local execution target and optional deployment config, then select +**Run and sign case**. Case parameters stay in the local fixture store outside +the workflow. Secret values are refused there and remain runner credential +references. Desktop retains the raw report locally and adds a signed receipt +containing its hash, outcome, required and passed contracts, evidence classes, +model-call count, and external-network-call count. + +Signed results from another customer-controlled qualification runner can be +imported. The signature, workflow revision, environment, runner capabilities, +and evidence hashes must match the project. + +## Certify and publish an immutable version + +When action review, identity coverage, effect coverage, and required cases all +pass, choose **Run certification**. Every refusal remains visible with the exact +action and contract that must be resolved. + +The artifact controls then provide the normal production lifecycle: + +1. **Create working version** preserves the previous revision. +2. **Seal and encrypt version** creates a protected candidate and stores only + its key reference in the operating-system credential store. +3. Run the required cases and certify that exact sealed version. +4. **Export qualified artifact** creates a deterministic archive with a + SHA-256 digest, or **Deploy qualified artifact** prepares the reviewed, + sanitized Cloud path. + +Raw recordings, reports, screenshots, case inputs, and secret values remain +inside the declared local boundary. Deployment never turns a pending local +review into an upload. + +## Script the same contract + +The CLI exposes the same versioned project: + +```bash +openadapt flow qualify init bundle \ + --target citrix \ + --application Accuro \ + --application-version 2026.1 \ + --environment-digest "$QUALIFIED_ENVIRONMENT_SHA256" \ + --minimum-tier 3 + +openadapt flow qualify inspect bundle --policy clinical-write +openadapt flow qualify set-risk bundle \ + --step step_012 \ + --classification consequential \ + --explanation "Writes the appointment record" +openadapt flow qualify explain bundle --policy clinical-write +openadapt flow qualify certify bundle \ + --policy clinical-write \ + --evidence-root qualification-evidence +``` + +Use `openadapt flow qualify schema` for the machine-readable project schema and +[`openadapt flow qualify --help`](../reference/cli.md#qualify) for identity, +effect, case, runner-trust, requalification, report, and certification commands. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 563f74c..214eef9 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -25,6 +25,7 @@ is a subcommand of `openadapt flow`. | [`teach`](#teach) | Resolve a halted run from a fix demonstration, governed | 0 if promoted, 1 if refused, 2 on bad inputs | | [`lint`](#lint) | Report a bundle's coverage gaps | nonzero by severity | | [`certify`](#certify) | Enforce a safety policy, refuse the bundle if it fails | 2 on failure | +| [`qualify`](#qualify) | Review, test, explain, and certify a versioned qualification project | nonzero on refusal | | [`disambiguate`](#disambiguate) | Surface and resolve compile-time ambiguities | 2 if a consequential ambiguity is unresolved | | [`connect`](#connect) | Pair this computer to a Cloud workspace (launcher command, needs OpenAdapt 1.7+) | 0/1 | | [`login`](#login) | Validate a hosted ingest token and remember the host | 0/1 | @@ -389,6 +390,58 @@ openadapt flow certify bundle --config deployment.yaml Provide `--policy` or a `--config` that sets `policy.policy`; certify errors if neither supplies a policy. Exits 2 when the bundle fails certification. +## qualify + +Create and operate a versioned qualification project for one compiled workflow, +application, execution surface, and environment. The commands below are the +scriptable counterpart of the Desktop qualification cockpit: they review action +risk, bind identity and effect contracts, run representative and fault cases, +explain refusals, and issue certification for the exact reviewed revision. + +```bash +openadapt flow qualify init bundle \ + --target citrix \ + --application Accuro \ + --application-version 2026.1 \ + --environment-digest "$QUALIFIED_ENVIRONMENT_SHA256" \ + --minimum-tier 3 + +openadapt flow qualify inspect bundle --policy clinical-write +openadapt flow qualify explain bundle --policy clinical-write +openadapt flow qualify certify bundle \ + --policy clinical-write \ + --evidence-root qualification-evidence +``` + +| Subcommand | What it does | +|---|---| +| `schema` | Print the machine-readable qualification project schema. | +| `init` | Bind a bundle to its target surface, application/version, environment, runtime, required runner capabilities, and minimum verification tier. | +| `inspect` | Show graph, action inventory, coverage, case state, requalification conditions, and certification readiness. | +| `set-risk` | Assign `read_only`, `state_changing`, `consequential`, or `irreversible` to one action, with an explanation. | +| `set-identity` | Arm an action with the canonical identity ladder or explicit signals, regions, matching rules, and quorum. | +| `set-effect` | Set the required verification tier for one declared effect. | +| `trust-runner` | Trust a qualification runner's signing key for imported case receipts. | +| `add-case` | Add a representative or fault case and its expected precise outcome. | +| `run` | Import and validate signed case-result receipts against the current workflow revision and environment. | +| `add-requalification` | Record an application, environment, workflow, policy, runtime, expiry, or operator-triggered requalification condition. | +| `explain` | Explain every certification refusal and the action needed to resolve it. | +| `report` | Generate the qualification report, including versions, action/risk inventory, identity and effect coverage, cases, exclusions, capabilities, hashes, and requalification conditions. | +| `certify` | Certify the exact project revision when its required contracts and cases pass. | + +`init --target` accepts `web`, `windows`, `macos`, `linux`, `rdp`, or +`citrix`. `add-case --kind` accepts `representative`, `ambiguity`, +`wrong_identity`, `stale_identity`, `weak_effect`, or `missing_effect`; its +expected outcome is one of `verified`, `completed_unverified`, `halted`, +`failed`, or `rolled_back`. Use each subcommand's `--help` for its complete +identity-signal, screen-region, effect, and runner-signing options. + +Qualification does not copy secret values into the bundle. Case inputs and raw +evidence remain in the local evidence root; imported receipts are accepted only +when their signatures, environment, revision, capabilities, and evidence hashes +match the project. See [Qualify a workflow](../guides/qualify-a-workflow.md) for +the complete Desktop and CLI journey. + ## disambiguate Surface the compile-time multiple-choice questions an ambiguous demonstration diff --git a/mkdocs.yml b/mkdocs.yml index 4eb8a21..118b6a3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -135,6 +135,7 @@ nav: - Parameters and secrets: guides/parameters-and-secrets.md - The capability ladder: concepts/capability-ladder.md - Write and enforce a policy: guides/policy-and-certification.md + - Qualify a workflow: guides/qualify-a-workflow.md - Policy and certify: concepts/policy-and-certify.md - Effect verification: concepts/effect-verification.md - The identity gate: concepts/identity-gate.md