Task — engineering-spec@1

Skill quality floor - NFR stubs, untrusted-content backport, pair parity

doneTASK-SKILL-202
module skill · class product · priority p1 · created 2026-07-23 · shipped null
depends on none · blocks none

TASK-SKILL-202: Skill quality floor - NFR stub disposition, untrusted-content backport, full pair parity

Summary

Three quality gaps ship in every consumer payload: four ~20-line NFR skill stubs are vendored as if they were real skills (no contract, no HITL rules - an agent routed to them will improvise "NFR certification" with false confidence); 24 of the 56 vendored skills - including the highest-exposure repo-readers - carry neither the untrusted_inputs frontmatter contract nor a references/UNTRUSTED_CONTENT.md, so prompt-injection discipline is absent exactly where arbitrary consumer-repo text is read; and the pair-parity checker holds only 11 of the 25 vendored author/audit pairs to its file-class floor. This task delists the stubs, backports the injection discipline from the exemplary skills to the 20 non-stub gaps, expands parity SCOPE to all pairs, and adds a skill-floor lint so undersized skills can never be silently vendored again.

Problem

Audit findings H7 + H8, verified first-hand 2026-07-23:

  1. Stub skills vendored as real (H7). build.sh:91-94 vendors nfr-certification-author, nfr-evaluator, nfr-test-runner, nfr-regression-handler - each a 20-22 line SKILL.md with a description and one paragraph, no envelopes, no invariants, no acceptance material. certify-nfrs.md routes to them. The chain-coverage checker exempts them as "unpaired by design", which answers the pairing question while leaving the deeper one - they are not skills, they are name reservations shipped as product.
  2. Injection discipline missing on 24/56 (H8). The repo's own best skills (task-author, task-audit) carry a two-part discipline: untrusted_inputs frontmatter (wrap-marker, injection scan, surface-to-human) and a references/UNTRUSTED_CONTENT.md procedure. 24 vendored skills carry neither - including repo-context-map-author and edge-case-matrix-author, which exist specifically to read arbitrary consumer-repo files, the canonical injection vector.
  3. Parity floor covers 11 of 25 pairs. check-pair-parity.sh SCOPE (:13-14) names 11 pairs; the payload carries 25. The 14 outside SCOPE can ship without envelopes, rubrics, or failure-mode references and nothing fails.

Proposed Solution

Delist the four NFR stubs from build.sh's VENDORED_SKILLS and remove their chain-allowlist.txt exemptions (the allowlist's own rot-warning rule then keeps the file honest); the source dirs remain in modules/skill/ as unvendored scaffolds. Give certify-nfrs.md an explicit "NFR skills not yet shipped - this workflow requires their full implementation" notice at its routing step so the workflow degrades loudly, not improvisationally. Backport the injection discipline to the 20 remaining gap skills by instantiating the task-author pattern: an untrusted_inputs frontmatter block (wrap_in_marker untrusted_content, injection_scan required, on_marker_hit surface_to_human) plus a references/UNTRUSTED_CONTENT.md adapted to each skill's actual input surface (repo files for the repo-readers; artefact bodies for the audits). Expand check-pair-parity.sh SCOPE to all 25 vendored pairs, authoring the missing file classes for the 14 newly-scoped pairs (envelopes, PIPELINE/INVARIANTS or RUBRIC/AUDIT_LOOP/REPORT_FORMAT, FAILURE_MODES, acceptance/README) at parity with the existing deepened pairs. Add tools/install/check-skill-floor.sh - a vendoring-time lint asserting every vendored SKILL.md meets a minimum floor (>= 60 body lines below frontmatter AND required sections present: a contract/usage heading, an operating-principles or MUST/MUST-NOT block; threshold chosen to be far below any real skill, far above any stub) - wired into build.sh so an undersized skill fails the build, with a test suite covering positive/negative paths.

Alternatives Considered

Success Metrics

Scope

In scope: build.sh delisting + floor-checker wiring, chain-allowlist cleanup, certify-nfrs.md notice, 20 injection-discipline backports, 14 pair deepenings + SCOPE expansion, the new checker + test suite, CHANGELOG.

Out of scope / Non-Goals

Dependencies

None blocking. TASK-SKILL-118 (done) established the pair file-class policy and the parity checker this task expands; TASK-CUO-209 (done) vendored the NFR stubs this task delists - both are context, neither needs reopening; their decisions are superseded knowingly and the delisting names TASK-CUO-209 in its CHANGELOG line. TASK-IMP-140's G7/G8 gates run this task's checkers in CI - forward reference only.

AI Authorship Disclosure

1. Description (normative)

2. Acceptance criteria

3. Edge cases

Audit

§1 — Verdict summary

Seven §1 clauses, seven ACs, six edge cases including a security-class row. The audit pressure fell on measurement honesty (plan's 21/24 vs measured 24/25 - corrected to measured values with the discrepancy recorded), degradation loudness for the delisted workflow surface, and anti-cargo-cult mechanics in the backport (byte-copy detection). Two count-drift traps were converted from counts to properties so the checkers stay true as the corpus moves.

§2 — Findings (all resolved)

ISS-001 — plan figures (21 skills, 24 pairs) did not match the measured corpus

Fresh measurement against the built payload found 24 skills missing both discipline halves and 25 author/audit pairs. Writing the plan's numbers into normative clauses would ship assertions that fail against reality on day one. Resolved: clauses and metrics use measured values; source_decisions records the plan-vs-measured discrepancy; AC 4 and t04 key on "SCOPE equals the payload's measured pair set" (a property), not a literal 25.

ISS-002 — delisting silently broke certify-nfrs.md in the first draft

Removing the four skills from the payload without touching the workflow that routes to them converts silent improvisation into a missing-skill error at best - or an agent improvising anyway at worst. Resolved: clause 1.2 requires the loud not-yet-shipped notice at the routing step; AC 2 asserts it; the delist-not-implement default is a recorded decision for the review gate.

ISS-003 — chain-allowlist entries would have become rot

chain-allowlist.txt's own header says an entry no payload dir matches triggers a rot warning; delisting without cleaning the four UNPAIRED lines would trade one checker's noise for another's. Resolved: clause 1.1 couples the two removals; AC 1 asserts the chain-coverage checker stays green.

ISS-004 — the backport could be satisfied by byte-copying one reference doc 20 times

The discipline's value is per-skill input-surface specificity; a generic copy passes a naive existence check while adding no protection. Resolved: clause 1.3 requires each doc to name the skill's own input surface; AC 3's test compares docs pairwise for full-content identity (identical files fail, shared sections pass).

ISS-005 — SCOPE expansion without the file classes would turn the parity checker red repo-wide

Expanding SCOPE to 25 pairs while 14 lack the class files makes the checker fail on main - a self-inflicted red that would get the expansion reverted. Resolved: clause 1.4 couples SCOPE expansion to authoring the missing classes "at parity with the existing deepened pairs, not as empty placeholder files"; AC 4 asserts pass on the payload AND the single-file-deletion negative.

ISS-006 — the stub-floor threshold needed a rationale and an exemption path

An arbitrary "60 lines" invites both gaming (padding) and legitimate-small-skill friction. Resolved: the floor pairs size with required-section structure (Alternatives rejects line-count-only), and the edge case defines the explicit, reviewed exemption path starting empty - the escape is visible, not improvised.

ISS-007 — placeholder detection overlap with existing tooling was unstated

TASK-SKILL-115's sweep-placeholders tooling already owns placeholder-syntax detection; a floor checker that duplicated it would create two authorities. Resolved: clause 1.5 scopes the floor to size + structure and names the existing sweep as the placeholder authority.

§3 — TRACE-006 semantic sufficiency (per clause)

ClauseVerb demandCited test assertsVerdict
1.1 MUST NOT vendor the four + MUST drop allowlist linespayload absence + allowlist absence + no rot warningAC 1: asserts all threesufficient
1.2 MUST state not-yet-shipped at routing stepnotice substring at the right stepAC 2: asserts presencesufficient
1.3 MUST carry both discipline halves, per-skillfrontmatter keys + non-empty per-skill doc + non-byte-copyAC 3: asserts all three per the 20 enumerated skillssufficient after revision (ISS-004)
1.4 MUST enumerate every pair + classes presentexit 0 on payload + SCOPE completeness + deletion negativeAC 4: asserts all threesufficient after revision (ISS-005)
1.5 MUST fail under-floor skills and fail the buildpass real payload, fail two fixture classes, fail injected buildAC 5: asserts all foursufficient
1.6 MUST cover the six behaviors, glob-registeredsuite green under run_all discoveryAC 6: asserts registration; t01-t05 are the behaviorssufficient
1.7 MUST record delisting/backport/expansionthree substrings in top entryAC 7: asserts all threesufficient

§4 — Resolution

Seven findings - one measurement-truth, six material - all resolved in the audited revision. Score = 10/10.

Status transition draft -> ready_to_implement is authorised by this verdict per STATUS-REFERENCE.md §1.1. The two human-acceptance gates in /ship-tasks are unchanged - this audit clears the spec-correctness gate only.


End of TASK-SKILL-202 audit.