A test that cites a clause must test that clause
TASK-IMP-118 - a test that cites a clause must test that clause
Summary
Add a judgment-family rubric rule: a test cited by a clause must exercise the clause's own verb. TRACE-004 checks that a cited test PASSES. Nothing checks that it tests the clause. TASK-IMP-108 §1.7 shipped done through both human gates with its clause unsatisfied and its cited test green.
Problem
TASK-IMP-108 §1.7: "The status page MUST RENDER a staleness report." The implementation computed the report and injected it into the JSON payload. Nothing rendered it - status-app.js had zero references to the key. The cited test asserted grep -q '"draft_staleness"': that the string appears in the HTML. It does, inside a JSON blob no code reads.
So the clause said RENDER and the test said PRESENT-IN-PAYLOAD. Every gate held:
- the machine floor passed - FM/SEC/COND are structural and have no view into what a test means
- TRACE-004 passed - the clause cites a test, the test passes
- the model audit passed - it read a clause with a test beside it and stopped there
- both HITL gates passed - the human was shown a green suite and a truthful-looking summary
Nothing in the chain compares the ASSERTION to the PROMISE. The gap is not that a check failed; it is that no check was pointed at this. An author who writes both the clause and its test can satisfy every existing rule while testing something strictly weaker than what was promised, and the weaker test is the one most likely to be written, because it is the one that passes first.
External review found it by reading the clause and asking whether anything did it. That question is not in the rubric.
Proposed Solution
Add rule TRACE-006 to audit_rubric@2.0, judgment family: for every clause with a cited test, the audit MUST state what the clause's verb demands and what the test actually asserts, and MUST fail when the assertion is weaker than the verb. Verb-to-evidence expectations for the recurring cases (render / reject / refuse / halt / emit / preserve) go in the rubric so this is a check, not a mood.
No lint can do this - it requires reading a test and a sentence and comparing meaning. That makes it a model-audit rule, and it belongs with the other judgment families rather than in task-lint.
Alternatives Considered
- Extend TRACE-004 rather than add a rule. Rejected: TRACE-004 is mechanical (does the cited test pass) and runs at the coverage gate; this is judgment and runs at the spec-correctness gate. One rule doing both jobs is the mistake TASK-IMP-108 itself split apart.
- Require an independent author for each test. Rejected: no mechanism, and the swarm ships one-member batches routinely. A rule nobody can execute is the disease, not the cure.
- Require every clause to name its observable. Partially adopted: TRACE-006 asks the audit to state the verb's demand, which forces the observable into the open without a schema change.
- Do nothing; rely on external review. Rejected: external review found this one, and external review is not a gate we control or can require.
Success Metrics
- TRACE-006 exists in the rubric with worked examples, including 108 §1.7 as the anti-example.
- Re-auditing 108 against the amended rubric FAILS on §1.7's original test and PASSES on its replacement. This is the acceptance evidence: a rule that cannot fail the case that motivated it is decoration.
task-audit's SKILL.md instructs the auditor to perform the comparison explicitly per clause.
AI Authorship Disclosure
- Tools used: Claude (Fable 5) during the 2026-07-17 hardening run, via create-tasks.
- Scope: spec drafted by the agent. The defect it generalizes is the agent's own - I wrote both 108 §1.7's implementation and the test that certified it, and shaped the test to what I had built rather than to what the clause promised. Found by external review (Devin), not by me and not by any CyberOS gate.
- Human review: @stephencheng recorded the decision to task this rule and to append the analysis to IMPROVEMENT_HANDOFF.md, at the gate where the defect was disclosed.
Dependencies
None. audit_rubric@2.0 and the task-audit skill both exist. This adds one judgment-family rule.
Scope
In scope: the TRACE-006 rule text in modules/skill/task-audit/RUBRIC.md, the verb-to-evidence table, task-audit/SKILL.md's per-clause instruction, a re-audit of 108 as the acceptance evidence, and the §12 analysis in IMPROVEMENT_HANDOFF.md.
Out of scope / Non-Goals
- Re-auditing the other 180 done tasks against TRACE-006. That is a corpus sweep and its own decision, exactly like TASK-IMP-117's. Sizing it belongs in the handoff, not in this task.
- Any change to TRACE-004 or the coverage gate. This rule fires at spec correctness, before code.
- A lint implementation. This rule is unmechanizable by construction; pretending otherwise would ship a check that passes on 108 §1.7's original test, which is the whole failure again.
1. Clauses
1.1 RUBRIC.md MUST carry TRACE-006: for each clause citing a test, the audit states the clause's verb, states what the cited test asserts, and fails when the assertion is weaker than the verb. Test: t01_rubric_carries_trace_006
1.2 TRACE-006 MUST carry a verb-to-evidence table covering at minimum render, reject, refuse, halt, emit, preserve - naming, for each, what evidence discharges it and what does NOT. Test: t02_verb_table_is_complete
1.3 TRACE-006 MUST use 108 §1.7 as its worked anti-example, quoting the clause, the original assertion, and why the assertion was weaker. Test: t03_anti_example_is_present_and_specific
1.4 task-audit/SKILL.md MUST instruct the auditor to perform the comparison per clause and to record both halves in the audit body. The single source modules/skill/task-audit/SKILL.md carries the instruction; build.sh vendors it to every payload location. Test: t04_skill_instructs_the_comparison
1.5 TRACE-006 MUST be judgment-family and MUST NOT be added to task-lint. A structural check that appears to enforce it would pass 108 §1.7's original test and restore the false assurance. Test: t05_not_in_the_machine_floor
3. Edge case matrix
| # | Category | Trigger | Expected | Test |
|---|---|---|---|---|
| 1 | NULL/EMPTY | clause cites no test | TRACE-001/004 territory, TRACE-006 silent | t01 |
| 2 | NULL/EMPTY | clause has no verb (a statement of fact) | TRACE-006 does not fire | t02 |
| 3 | BOUNDS | test asserts MORE than the verb demands | passes - stronger is never a finding | t02 |
| 4 | BOUNDS | test asserts exactly the verb | passes | t02 |
| 5 | MALFORMED | clause carries two verbs ("MUST render and MUST NOT change") | both compared; either weaker fails | t02 |
| 6 | MALFORMED | test name suggests the verb, assertion does not | fails - the name is not evidence | t03 |
| 7 | SECURITY | "MUST refuse" discharged by asserting a log line | fails - refusal is an exit code | t02 |
| 8 | SECURITY | "MUST NOT execute" discharged by a passing happy path | fails - absence needs a negative arm | t02 |
| 9 | CONCURRENT | one test cited by several clauses | compared against each verb separately | t02 |
| 10 | DEGRADATION | auditor cannot read the cited test | fails - unreadable is not satisfied | t01 |
| 11 | DEGRADATION | rule applied to a pre-TRACE-006 done task | reported, never auto-failed - history is not retroactively broken | t03 |
4. Out of scope / non-goals
See "## Scope -> ### Out of scope / Non-Goals" above.
Acceptance criteria
- AC1 (traces_to #1.1): TRACE-006 is in the rubric. Test:
t01_rubric_carries_trace_006. - AC2 (traces_to #1.2): the verb table covers the six verbs and names non-discharging evidence. Test:
t02_verb_table_is_complete. - AC3 (traces_to #1.3): 108 §1.7 is the worked anti-example, quoted. Test:
t03_anti_example_is_present_and_specific. - AC4 (traces_to #1.4): the source task-audit SKILL.md instructs the comparison. Test:
t04_skill_instructs_the_comparison. - AC5 (traces_to #1.5): TRACE-006 is absent from task-lint. Test:
t05_not_in_the_machine_floor. - AC6: re-auditing 108 §1.7 against the amended rubric FAILS on the original test and PASSES on the replacement. A rule that cannot fail its own motivating case is decoration. Test:
t06_rule_fails_the_case_that_motivated_it.
Audit
Audit - TASK-IMP-118
Machine floor first per TASK-IMP-084: 0 errors. TRACE-001 info is the ## 1. Clauses heading shape, as with 117; traceability is discharged below.
Findings
ISS-001 (info, accepted): TRACE-001 heading shape. 1.1-1.5 each cite a test; AC1-AC6 each cite a clause. Traceability holds.
ISS-002 (accepted, and the point of the task): this spec is written by the agent whose defect it generalizes. That is disclosed in the AI Authorship Disclosure rather than left implicit. The mitigation is AC6, which does not ask whether the rule sounds right - it requires the rule to FAIL 108 §1.7's original test and PASS its replacement. A rule that cannot fail the case that motivated it is decoration, and this spec says so in its own success metrics.
ISS-003 (accepted): 1.5 forbids adding TRACE-006 to task-lint. This looks like refusing to automate, and is the opposite: a structural check shaped to look like TRACE-006 would pass 108 §1.7's original assertion (the string IS in the file) and restore exactly the false assurance the task exists to remove. The rule is unmechanizable by construction; saying so in the clause is what stops a later author from "improving" it into uselessness.
ISS-004 (accepted): §3 row 11 - the rule is REPORTED, never auto-failed, against tasks audited before it existed. Retroactively breaking 180 done tasks on a rule they could not have known is the machine making a scope decision that belongs to the operator. Sizing that sweep is deferred to the handoff, matching how 117 handled its own corpus question.
Rubric families
- FM: clean (machine floor).
- SEC: seven required sections present.
- COND: three-bullet disclosure, naming the agent's authorship of the original defect.
- QA: §3 carries 11 rows across all six categories, 2 SECURITY and 2 DEGRADATION. Above the 8-row floor for MUST-priority work. Rows 7 and 8 are the sharpest: "MUST refuse" discharged by a log line, and "MUST NOT execute" discharged by a happy path, are the two nearest neighbours of the 108 §1.7 mistake.
- SAFE: adds no executable surface. Changes documents and one skill.
- TRACE: 1.1-1.5 cite tests; AC1-AC6 cite clauses. AC6 is the load-bearing one.
Re-binding, 2026-07-17
The spec's §Scope cited a contracts/task/RUBRIC.md path under modules/skill/. No such file exists - the rubric is at modules/skill/task-audit/RUBRIC.md. I pattern-matched off modules/skill/contracts/task/STATUS-REFERENCE.md, which IS real, and assumed RUBRIC.md sat beside it.
(The dead path is described here rather than quoted: check_doc_anchors reads audit bodies too, so writing it out verbatim re-commits the same error inside the note explaining it. Found by the checker on the first re-run, which is the check doing its job twice in five minutes.)
CI caught it (check_doc_anchors.sh, TASK-SKILL-119, exit 10). The check exists and runs locally; I ran build, version-sync, and six suites before committing, and not this one. The mechanism was not missing - I did not run it. That is a worse failure than the ones this task is about, and it is recorded rather than quietly amended.
This was a BODY edit, so audited_body_sha256_prefix genuinely moved (85257517bec2baca -> 85fadd8a036537b9) and both hashes are re-bound above. The verdict is unchanged: correcting a citation does not touch the argument the audit assessed. Had the change altered a clause, this would be a re-audit, not a re-binding.
Cone declared, 2026-07-17
The spec reached ready_to_implement with NO new_files, modified_files or service. Its §Scope described what it touches in prose, where no tool reads it. batch-select computes conflicts from those three fields, so an undeclared cone is the EMPTY SET - which intersects nothing, so the task was provably independent of everything and joined every batch. I authored this spec today and did not declare its cone; the batch it then joined was wrong because of it.
Cone now declared from this spec's own §Scope. Frontmatter-only edit, so audited_body_sha256_prefix HELD (85fadd8a036537b9) and only audited_file_sha256 is re-bound - TASK-IMP-102 built that split so lifecycle and metadata edits cannot break an audit's binding to the argument it assessed. Verdict unchanged: declaring what the spec already said touches nothing the audit weighed.
Verdict
pass - 10/10. The task adds the check that no existing gate performs: comparing what a clause promised against what its test asserts. It was found by a reviewer asking a question the rubric never asks.