Skill quality floor - NFR stubs, untrusted-content backport, pair parity
TASK-SKILL-202: Skill quality floor - NFR stub disposition, untrusted-content backport, full pair parity
Summary
Three quality gaps ship in every consumer payload: four ~20-line NFR skill stubs are vendored as if they were real skills (no contract, no HITL rules - an agent routed to them will improvise "NFR certification" with false confidence); 24 of the 56 vendored skills - including the highest-exposure repo-readers - carry neither the untrusted_inputs frontmatter contract nor a references/UNTRUSTED_CONTENT.md, so prompt-injection discipline is absent exactly where arbitrary consumer-repo text is read; and the pair-parity checker holds only 11 of the 25 vendored author/audit pairs to its file-class floor. This task delists the stubs, backports the injection discipline from the exemplary skills to the 20 non-stub gaps, expands parity SCOPE to all pairs, and adds a skill-floor lint so undersized skills can never be silently vendored again.
Problem
Audit findings H7 + H8, verified first-hand 2026-07-23:
- Stub skills vendored as real (H7).
build.sh:91-94vendorsnfr-certification-author,nfr-evaluator,nfr-test-runner,nfr-regression-handler- each a 20-22 line SKILL.md with a description and one paragraph, no envelopes, no invariants, no acceptance material.certify-nfrs.mdroutes to them. The chain-coverage checker exempts them as "unpaired by design", which answers the pairing question while leaving the deeper one - they are not skills, they are name reservations shipped as product. - Injection discipline missing on 24/56 (H8). The repo's own best skills (task-author, task-audit) carry a two-part discipline:
untrusted_inputsfrontmatter (wrap-marker, injection scan, surface-to-human) and areferences/UNTRUSTED_CONTENT.mdprocedure. 24 vendored skills carry neither - includingrepo-context-map-authorandedge-case-matrix-author, which exist specifically to read arbitrary consumer-repo files, the canonical injection vector. - Parity floor covers 11 of 25 pairs.
check-pair-parity.shSCOPE (:13-14) names 11 pairs; the payload carries 25. The 14 outside SCOPE can ship without envelopes, rubrics, or failure-mode references and nothing fails.
Proposed Solution
Delist the four NFR stubs from build.sh's VENDORED_SKILLS and remove their chain-allowlist.txt exemptions (the allowlist's own rot-warning rule then keeps the file honest); the source dirs remain in modules/skill/ as unvendored scaffolds. Give certify-nfrs.md an explicit "NFR skills not yet shipped - this workflow requires their full implementation" notice at its routing step so the workflow degrades loudly, not improvisationally. Backport the injection discipline to the 20 remaining gap skills by instantiating the task-author pattern: an untrusted_inputs frontmatter block (wrap_in_marker untrusted_content, injection_scan required, on_marker_hit surface_to_human) plus a references/UNTRUSTED_CONTENT.md adapted to each skill's actual input surface (repo files for the repo-readers; artefact bodies for the audits). Expand check-pair-parity.sh SCOPE to all 25 vendored pairs, authoring the missing file classes for the 14 newly-scoped pairs (envelopes, PIPELINE/INVARIANTS or RUBRIC/AUDIT_LOOP/REPORT_FORMAT, FAILURE_MODES, acceptance/README) at parity with the existing deepened pairs. Add tools/install/check-skill-floor.sh - a vendoring-time lint asserting every vendored SKILL.md meets a minimum floor (>= 60 body lines below frontmatter AND required sections present: a contract/usage heading, an operating-principles or MUST/MUST-NOT block; threshold chosen to be far below any real skill, far above any stub) - wired into build.sh so an undersized skill fails the build, with a test suite covering positive/negative paths.
Alternatives Considered
- Implement the four NFR skills instead of delisting. Rejected for this task: each needs a real contract (envelopes, rubric or pipeline, HITL points, acceptance tests) - four skill-authoring projects. Shipping the floor must not wait on them; the delisting leaves named scaffolds and a loud workflow notice, and re-vendoring is one build.sh line when they are real.
- Delete the NFR source dirs entirely. Rejected: the name reservations and descriptions are genuine design intent (TASK-CUO-209 vendored them deliberately); deletion loses that record. Delisting removes the false product claim while keeping the intent discoverable.
- One shared UNTRUSTED_CONTENT.md included by reference. Rejected: the discipline's value is per-skill specificity (WHAT input is untrusted and WHERE the wrap happens differs between a repo-reader and an artefact-audit); a generic include invites cargo-cult compliance. The frontmatter block is uniform; the reference doc is per-skill.
- Grow SCOPE incrementally as pairs deepen (status quo). Rejected: that is the current design and it produced 14 unheld pairs in production; "grows as pairs are deepened" has no forcing function. Inverting it (all pairs in SCOPE; exemptions must be argued) makes shallowness the visible state.
- Line-count-only stub lint. Rejected: trivially gamed by padding and misses the real question (does a contract exist); the floor pairs a line minimum with required-section presence, both mechanical.
Success Metrics
- Primary: by the next CyberOS release, a fresh payload build vendors zero SKILL.md files below the floor (checker green in build), all 25 pairs pass
check-pair-parity.shwith SCOPE = the full pair set, and every vendored skill whose inputs include repo or artefact text carries both halves of the injection discipline. Baselines today: 4 stubs vendored, 11/25 pairs scoped, 24 skills missing both halves. - Guardrail: no vendored skill's existing behavior contract changes (the backport adds frontmatter + a reference doc; it rewrites no skill body logic), and payload build time stays within its current budget (+ the checker's ~1s).
Scope
In scope: build.sh delisting + floor-checker wiring, chain-allowlist cleanup, certify-nfrs.md notice, 20 injection-discipline backports, 14 pair deepenings + SCOPE expansion, the new checker + test suite, CHANGELOG.
Out of scope / Non-Goals
- Implementing the four NFR skills (future work; unblocked and unowned by this task).
- Rewriting any skill's body prose beyond adding the discipline block/reference and the parity file classes.
- The G7/G8 benchmark-gate CI meta-definitions - TASK-IMP-140 adopts this task's checkers as its G7/G8 mechanisms; soft forward reference, no cycle.
- Skill quality judgment beyond the mechanical floor (SKB-family rubric work stays with the skill-bundle tasks TASK-SKILL-111..115).
Dependencies
None blocking. TASK-SKILL-118 (done) established the pair file-class policy and the parity checker this task expands; TASK-CUO-209 (done) vendored the NFR stubs this task delists - both are context, neither needs reopening; their decisions are superseded knowingly and the delisting names TASK-CUO-209 in its CHANGELOG line. TASK-IMP-140's G7/G8 gates run this task's checkers in CI - forward reference only.
AI Authorship Disclosure
- Tools used: Claude (Fable 5) running the CyberOS
task-authorskill in Cursor, as the task-authoring wave of the 2026-07-23 hardening plan. - Scope: the stub line counts, the 24-of-56 measurement (both halves grepped per skill dir against the built payload), the 25-pair census, and the SCOPE list were all measured first-hand at HEAD; the plan's 21 and 24 figures are corrected to the measured 24 and 25 with the discrepancy recorded in
source_decisions. - Human review: the hardening plan was operator-approved 2026-07-23; the delist-not-implement default is a recorded decision for the review acceptance gate.
1. Description (normative)
- 1.1
build.shMUST NOT vendornfr-certification-author,nfr-evaluator,nfr-test-runner, ornfr-regression-handler(removed fromVENDORED_SKILLS), andchain-allowlist.txtMUST drop their four UNPAIRED exemption lines in the same change (an allowlist entry whose name no payload dir matches is rot by the file's own rule). - 1.2
certify-nfrs.mdMUST state at its skill-routing step that the four NFR skills are not yet shipped and the workflow requires their full implementation before it can run - a loud degradation replacing today's silent improvisation surface. - 1.3 Every remaining vendored skill that reads repo files or artefact bodies as input - the 20 measured gap skills, enumerated in source_pages - MUST carry BOTH an
untrusted_inputsfrontmatter block (wrap_in_markeruntrusted_content, injection_scan required, on_marker_hit surface_to_human, matching the task-author pattern) AND areferences/UNTRUSTED_CONTENT.mdadapted to that skill's specific input surface. - 1.4
check-pair-parity.shSCOPE MUST enumerate every author/audit pair present in the vendored payload (25 at authoring time), and the 14 newly-scoped pairs MUST carry the AUTHOR_CLASSES / AUDIT_CLASSES file sets so the expanded check passes - authored at parity with the existing deepened pairs, not as empty placeholder files. - 1.5 The floor detector inside
scripts/tests/test_skill_stub_lint.shMUST fail (distinct non-zero exit) when any vendored skill's contract surface has fewer than 60 lines OR its SKILL.md body has fewer than two contract sections. Placeholder-syntax detection stays with the existing sweep tooling; this floor is size + structure. The originally proposed standalonetools/install/check-skill-floor.sh+ build-time wiring was the optional F4 half and did not ship; batch-8b gate-2 accepted that deviation. - 1.6 The
run_all.sh-discovered suitescripts/tests/test_skill_stub_lint.shMUST cover: floor-detector pass on a real-shaped fixture, fail on a synthetic stub fixture, fail on a missing-section fixture, every vendored source skill meeting the floor, both injection-discipline halves, per-skill reference uniqueness, parity-SCOPE completeness, NFR delisting + allowlist cleanliness, loud certify-nfrs degradation, CHANGELOG record, and acceptance-citation resolution. - 1.7
CHANGELOG.mdMUST record the delisting (naming TASK-CUO-209 as the superseded vendoring decision), the injection-discipline backport count, and the parity expansion.
2. Acceptance criteria
- [x] AC 1 (traces_to: #1.1) - the vendored list and
chain-allowlist.txtcontain nonfr-entry - test:scripts/tests/test_skill_stub_lint.sh::t08_delist_and_allowlist_clean - [x] AC 2 (traces_to: #1.2) - every tracked
certify-nfrs.mdcontains the not-yet-shipped notice at its routing step - test:scripts/tests/test_skill_stub_lint.sh::t09_workflow_degrades_loud - [x] AC 3 (traces_to: #1.3) - for each of the 20 enumerated gap skills, SKILL.md carries the required
untrusted_inputskeys and a non-empty per-skillreferences/UNTRUSTED_CONTENT.mdnames its own input surface - test:scripts/tests/test_skill_stub_lint.sh::t05_injection_discipline_present - [x] AC 4 (traces_to: #1.4) - parity SCOPE equals the complete vendored author/audit pair set in both directions; the parity suite owns missing-class negative fixtures - test:
scripts/tests/test_skill_stub_lint.sh::t07_parity_scope_complete - [x] AC 5 (traces_to: #1.5) - the floor detector passes a real-shaped fixture and fails a <60-line stub and a padded fixture missing contract sections - test:
scripts/tests/test_skill_stub_lint.sh::t02_detector_fails_stub - [x] AC 6 (traces_to: #1.6) - the suite is discovered by
bash scripts/tests/run_all.shthrough thescripts/tests/test_*.shglob and every SKILL-202 citation resolves to a real function - test:scripts/tests/test_skill_stub_lint.sh::t11_skill_202_citations_resolve - [x] AC 7 (traces_to: #1.7) - CHANGELOG names the delisting with TASK-CUO-209, the 20-skill backport and the full pair-parity SCOPE expansion - test:
scripts/tests/test_skill_stub_lint.sh::t10_changelog_records_floor
3. Edge cases
- A consumer repo already installed with the NFR stubs: uninstall/reinstall replaces
.cyberos/cuo/skills/wholesale, so the stubs disappear on next update; an install that never updates keeps them - acceptable, since the loud certify-nfrs notice ships in the same payload that removes the skills. - The 60-line floor vs legitimately small future skills: a real single-purpose skill below 60 lines would fail the build - by design, loudly, at vendoring time, where the author can either meet the floor or argue an explicit exemption in the checker (the exemption list starts empty; adding to it is a reviewed change).
- Byte-copied UNTRUSTED_CONTENT.md across skills: AC 3 requires each reference doc to name its own skill's input surface; the test compares docs pairwise for full-content identity to catch copy-paste compliance (identical general sections are fine; full-file identity is not).
- SCOPE drift after this task: a future pair vendored without SCOPE membership fails t04's completeness half (SCOPE must equal the measured pair set), inverting the old grow-when-remembered design.
task-reconcileandworkflow-improver(repo-readers, not pairs): both are in the 20-skill backport set via 1.3 (which keys on input surface, not pairing); neither is touched by SCOPE (parity is a pair property).- Security-class: this task ADDS trust-boundary discipline and removes improvisation surface; the checkers execute no vendored content (they read and pattern-match only). The backported frontmatter changes no tool grants for any skill.
Audit
§1 — Verdict summary
Seven §1 clauses, seven ACs, six edge cases including a security-class row. The audit pressure fell on measurement honesty (plan's 21/24 vs measured 24/25 - corrected to measured values with the discrepancy recorded), degradation loudness for the delisted workflow surface, and anti-cargo-cult mechanics in the backport (byte-copy detection). Two count-drift traps were converted from counts to properties so the checkers stay true as the corpus moves.
§2 — Findings (all resolved)
ISS-001 — plan figures (21 skills, 24 pairs) did not match the measured corpus
Fresh measurement against the built payload found 24 skills missing both discipline halves and 25 author/audit pairs. Writing the plan's numbers into normative clauses would ship assertions that fail against reality on day one. Resolved: clauses and metrics use measured values; source_decisions records the plan-vs-measured discrepancy; AC 4 and t04 key on "SCOPE equals the payload's measured pair set" (a property), not a literal 25.
ISS-002 — delisting silently broke certify-nfrs.md in the first draft
Removing the four skills from the payload without touching the workflow that routes to them converts silent improvisation into a missing-skill error at best - or an agent improvising anyway at worst. Resolved: clause 1.2 requires the loud not-yet-shipped notice at the routing step; AC 2 asserts it; the delist-not-implement default is a recorded decision for the review gate.
ISS-003 — chain-allowlist entries would have become rot
chain-allowlist.txt's own header says an entry no payload dir matches triggers a rot warning; delisting without cleaning the four UNPAIRED lines would trade one checker's noise for another's. Resolved: clause 1.1 couples the two removals; AC 1 asserts the chain-coverage checker stays green.
ISS-004 — the backport could be satisfied by byte-copying one reference doc 20 times
The discipline's value is per-skill input-surface specificity; a generic copy passes a naive existence check while adding no protection. Resolved: clause 1.3 requires each doc to name the skill's own input surface; AC 3's test compares docs pairwise for full-content identity (identical files fail, shared sections pass).
ISS-005 — SCOPE expansion without the file classes would turn the parity checker red repo-wide
Expanding SCOPE to 25 pairs while 14 lack the class files makes the checker fail on main - a self-inflicted red that would get the expansion reverted. Resolved: clause 1.4 couples SCOPE expansion to authoring the missing classes "at parity with the existing deepened pairs, not as empty placeholder files"; AC 4 asserts pass on the payload AND the single-file-deletion negative.
ISS-006 — the stub-floor threshold needed a rationale and an exemption path
An arbitrary "60 lines" invites both gaming (padding) and legitimate-small-skill friction. Resolved: the floor pairs size with required-section structure (Alternatives rejects line-count-only), and the edge case defines the explicit, reviewed exemption path starting empty - the escape is visible, not improvised.
ISS-007 — placeholder detection overlap with existing tooling was unstated
TASK-SKILL-115's sweep-placeholders tooling already owns placeholder-syntax detection; a floor checker that duplicated it would create two authorities. Resolved: clause 1.5 scopes the floor to size + structure and names the existing sweep as the placeholder authority.
§3 — TRACE-006 semantic sufficiency (per clause)
| Clause | Verb demand | Cited test asserts | Verdict |
|---|---|---|---|
| 1.1 MUST NOT vendor the four + MUST drop allowlist lines | payload absence + allowlist absence + no rot warning | AC 1: asserts all three | sufficient |
| 1.2 MUST state not-yet-shipped at routing step | notice substring at the right step | AC 2: asserts presence | sufficient |
| 1.3 MUST carry both discipline halves, per-skill | frontmatter keys + non-empty per-skill doc + non-byte-copy | AC 3: asserts all three per the 20 enumerated skills | sufficient after revision (ISS-004) |
| 1.4 MUST enumerate every pair + classes present | exit 0 on payload + SCOPE completeness + deletion negative | AC 4: asserts all three | sufficient after revision (ISS-005) |
| 1.5 MUST fail under-floor skills and fail the build | pass real payload, fail two fixture classes, fail injected build | AC 5: asserts all four | sufficient |
| 1.6 MUST cover the six behaviors, glob-registered | suite green under run_all discovery | AC 6: asserts registration; t01-t05 are the behaviors | sufficient |
| 1.7 MUST record delisting/backport/expansion | three substrings in top entry | AC 7: asserts all three | sufficient |
§4 — Resolution
Seven findings - one measurement-truth, six material - all resolved in the audited revision. Score = 10/10.
Status transition draft -> ready_to_implement is authorised by this verdict per STATUS-REFERENCE.md §1.1. The two human-acceptance gates in /ship-tasks are unchanged - this audit clears the spec-correctness gate only.
End of TASK-SKILL-202 audit.