"Goldensets as first-class gate inputs"
TASK-IMP-008: Goldensets as first-class gate inputs
Summary
Make goldensets a real, documented gate input for the CyberOS 1.x install/payload surface: a sealed tools/install/.awh/goldenset.yaml + baseline, a runner that prefers awh and degrades cleanly when awh is absent, suite coverage, and a lightweight CI step alongside the existing module awh-gate.
Problem
Module goldensets exist under modules/*/.awh/ and feed awh-gate.yml, but the install/payload tooling that ships CyberOS 1.x has no first-class goldenset. awh_goldenset_from_task.py helps author module tasks; nothing standardizes an install-path goldenset or a skip-clean runner for hosts without awh.
Proposed Solution
- Author
tools/install/.awh/goldenset.yamlwith lightweight, offline-safe tasks (VERSION semver, help.sh, run-gates script presence/shebang check, coverage-ratchet --help). - Commit a matching
eval-baseline.jsonsealed from a green run. - Ship
tools/install/run-goldenset.shthat: runs viaawh evalwhen available; otherwise executes each taskcmdwith a tiny fallback runner; skips cleanly whenCYBEROS_SKIP_GOLDENSET=1or when neither awh nor python3 is available (documented SKIP). - Document at
docs/verification/install-goldenset.md. - Add an
install-goldensetjob toawh-gate.ymlthat does not require the docker/Postgres stack.
Alternatives Considered
- Only document module goldensets. Rejected: mission asks for payload-relevant install goldenset.
- Require awh always. Rejected: local macOS contributors may lack awh; skip-clean is mandatory.
- Put install goldenset under modules/install. Rejected: install lives in
tools/install; keep path honest.
Success Metrics
- Primary: suite proves skip / awh-or-fallback run / fail-closed without baseline when forced.
- Guardrail: existing module awh-gate job unchanged in spirit; install job is additive and offline-safe.
Scope
In scope: install goldenset + baseline + runner + docs + suite + awh-gate.yml job + CHANGELOG.
Out of scope / Non-Goals
- Regenerating every module goldenset.
- Making awh a hard install dependency.
- Auto-revert (TASK-IMP-026).
Dependencies
None hard. Soft: awh CLI / tools/awh harness when present.
AI Authorship Disclosure
- Tools used: Cursor agent, batch/12c.
- Human review: session operator Stephen Cheng.
§1 - Description (normative)
tools/install/.awh/goldenset.yamlMUST exist with schema compatible with awh (tasks:list; each task hasid,cmd,weight,timeout_sec).- At least three tasks MUST be offline-safe (no network, no docker, no cargo).
tools/install/.awh/eval-baseline.jsonMUST be committed; awh-gate install job MUST fail closed if goldenset exists without baseline.tools/install/run-goldenset.shMUST:
- exit 0 with a
SKIPline whenCYBEROS_SKIP_GOLDENSET=1; - prefer
awh eval … --baseline … --max-regression 0.0whenawhis on PATH orpython3 -m harness.cliworks withPYTHONPATH=tools/awh; - otherwise run each task
cmdvia a python3 YAML fallback; - if neither awh nor python3 is available, exit 0 with
SKIPnaming the reason.
docs/verification/install-goldenset.mdMUST document path, runner, skip rules, and CI wiring..github/workflows/awh-gate.ymlMUST gain aninstall-goldensetjob (no postgres/redis) that runs the install goldenset whentools/install/**changes (or always on PR — either is acceptable if documented).- Suite
test_install_goldenset.shMUST cover skip + runner presence + goldenset shape.
Acceptance criteria
- [x] AC1: goldenset.yaml parses and lists >= 3 tasks with id+cmd. (t01)
- [x] AC2: baseline file present next to goldenset. (t02)
- [x] AC3:
CYBEROS_SKIP_GOLDENSET=1→ exit 0 + SKIP. (t03) - [x] AC4: runner executes successfully on this host (awh path validates committed baseline; fallback executes committed tasks without baseline compare). (t04)
- [x] AC5: docs page names runner + skip env + CI job. (t05)
- [x] AC6: awh-gate.yml contains install-goldenset job referencing tools/install/.awh. (t06)
Test plan
bash tools/install/tests/test_install_goldenset.sh
Audit
TASK-IMP-008 audit
Install goldenset path is payload-honest (tools/install/.awh/). Fallback-or-SKIP when awh absent is normative; CI install-goldenset job is offline-safe and additive to module awh-gate. depends_on empty; IMP-026 consumes the standardized path.
SUMMARY verdict: pass issues_open: 0 next_action: none