Workshop file reviewed
workshop/27-evaluate-workflow-quality.md
Problem
The step introduces the evals: frontmatter block without noting that it is an experimental feature. The intro line states:
"You'll add an evals: block to your workflow, define binary quality questions, run the workflow, and verify that results are recorded in the evals artifact (evals.jsonl) and persisted for historical comparison."
This presents evals: as a settled, stable feature.
Current correct information
Per the gh-aw reference docs (syntax-agentic.md, describing the evals: frontmatter key):
"evals: - ⚠️ Experimental. BinEval binary (YES/NO) evaluation questions run after safe-outputs and before the conclusion job. Shorthand: a list of { id, question, model? } objects."
The ⚠️ Experimental caveat is meaningful for learners deciding whether to depend on this feature, since experimental fields can change or be removed in future gh-aw releases.
Suggested fix
Add a brief inline note or > [!NOTE] callout near the top of the step (e.g. right after "What You'll Do") stating that evals: is an experimental frontmatter key that may change in future gh-aw releases, consistent with the reference documentation.
Generated by 🔍 Workshop Sync Check · copilot · auto · 133.6 AIC · ⌖ 5.96 AIC · ⊞ 8.6K · ◷
Workshop file reviewed
workshop/27-evaluate-workflow-quality.mdProblem
The step introduces the
evals:frontmatter block without noting that it is an experimental feature. The intro line states:This presents
evals:as a settled, stable feature.Current correct information
Per the gh-aw reference docs (
syntax-agentic.md, describing theevals:frontmatter key):The⚠️ Experimental caveat is meaningful for learners deciding whether to depend on this feature, since experimental fields can change or be removed in future gh-aw releases.
Suggested fix
Add a brief inline note or
> [!NOTE]callout near the top of the step (e.g. right after "What You'll Do") stating thatevals:is an experimental frontmatter key that may change in future gh-aw releases, consistent with the reference documentation.