Task — engineering-spec@1

"CUO per-step rollback on chain failure — execute compensating actions in reverse order with partial-execution audit preserved"

doneTASK-CUO-105
module cuo · class product · priority p0 · created 2026-05-17 · shipped null
depends on TASK-CUO-104 · blocks none

§1 — Description (BCP-14 normative)

The CUO service MUST ship per-step rollback at services/cuo/src/rollback/ triggered by TASK-CUO-104 step failure, compensations in reverse order, immutable audit, 6 memory audit kinds.

  1. MUST validate rollback_step_status against closed enum per DEC-2371.
  1. MUST trigger on TASK-CUO-104 step failure (chain status=failed) per DEC-2370.
  1. MUST execute at executor.rs::rollback(chain_id) per DEC-2370:
  1. MUST lookup compensation at compensation_registry.rs::get(skill_id) per DEC-2372 — returns Option<compensating_skill_id>.
  1. MUST define table at migration 0005: ``sql CREATE TABLE cuo_chain_rollbacks ( rollback_id UUID PRIMARY KEY, tenant_id UUID NOT NULL, chain_id UUID NOT NULL REFERENCES cuo_chain_walks(chain_id), step_id UUID NOT NULL REFERENCES cuo_chain_steps(step_id), compensating_skill_id TEXT, status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending','compensating','compensated','no_compensation_registered','compensation_failed')), started_at TIMESTAMPTZ, completed_at TIMESTAMPTZ, failure_reason TEXT, trace_id CHAR(32), created_at TIMESTAMPTZ NOT NULL DEFAULT now(), UNIQUE (chain_id, step_id) ); CREATE INDEX rollbacks_chain_idx ON cuo_chain_rollbacks(tenant_id, chain_id); ALTER TABLE cuo_chain_rollbacks ENABLE ROW LEVEL SECURITY; CREATE POLICY rollbacks_rls ON cuo_chain_rollbacks USING (tenant_id = current_setting('auth.tenant_id')::uuid) WITH CHECK (tenant_id = current_setting('auth.tenant_id')::uuid); REVOKE UPDATE, DELETE ON cuo_chain_rollbacks FROM cyberos_app; GRANT UPDATE (status, started_at, completed_at, failure_reason) ON cuo_chain_rollbacks TO cyberos_app; ``
  1. MUST update TASK-CUO-104 chain status to partial_rolled_back post-completion per DEC-2373.
  1. MUST expose endpoints: ``text POST /v1/cuo/chains/{id}/rollback (auto on step failure; CDO manual trigger) GET /v1/cuo/chains/{id}/rollback-status (per-step status) ``
  1. MUST emit 6 memory audit kinds per DEC-2374. PII per TASK-MEMORY-111: failure_reason SHA256.
  1. MUST thread trace_id from chain failure → rollback → audit.
  1. MUST NOT skip compensation if registered per DEC-2370.
  1. MUST NOT rollback in forward order per DEC-2370 (always reverse).

§2 — Why this design

Why reverse order (DEC-2370)? Compensations undo state changes; must unwind in reverse to handle dependencies between steps.

Why no-compensation preservation (DEC-2372)? Some operations (sending email, charging card) have no safe undo; preserving + auditing is the only correct option.

Why immutable rollback rows (DEC-2373)? Audit lineage; rollback is a financial-grade operation.


§3 — API contract

Sample rollback status:

{
  "rollback_id": "uuid",
  "chain_id": "uuid",
  "steps": [
    {"step_order": 2, "skill_id": "email.send", "status": "no_compensation_registered"},
    {"step_order": 1, "skill_id": "calendar.book", "compensating_skill_id": "calendar.cancel", "status": "compensated"},
    {"step_order": 0, "skill_id": "auth.create_invite", "compensating_skill_id": "auth.revoke_invite", "status": "compensated"}
  ]
}

§4 — Acceptance criteria

  1. rollback_step_status enum cardinality 5. 2. Triggered on TASK-CUO-104 step failure. 3. Reverse order execution. 4. Compensation looked up from TASK-SKILL-001 registry. 5. Missing compensation preserved with audit (not failure). 6. 6 memory audit kinds emitted. 7. PII scrubbed (failure_reason SHA256). 8. RLS denies cross-tenant. 9. Trace_id preserved. 10. UNIQUE(chain_id, step_id). 11. Chain status → partial_rolled_back post. 12. Append-only via REVOKE except status cols. 13. CDO manual trigger allowed. 14. Per-step compensation isolated (one failure doesn't halt rollback). 15. compensation_failed audited sev-2. 16. Idempotent (re-trigger uses existing rollback_id per step). 17. Rollback of failed step itself skipped (not completed). 18. Skipped steps (TASK-CUO-104) also no-rollback. 19. Order matches inverse of step_order. 20. Rollback timeout per step 30s.

§5 — Verification

#[tokio::test]
async fn rollback_in_reverse_order() {
    let ctx = TestContext::with_3_step_chain_step_3_fails().await;
    ctx.run_chain(ctx.chain_id).await;
    let rollbacks = ctx.fetch_rollbacks(ctx.chain_id).await;
    let order: Vec<i32> = rollbacks.iter().map(|r| r.step_order).collect();
    assert_eq!(order, vec![2, 1, 0]);  // reverse
}

#[tokio::test]
async fn no_compensation_preserved() {
    let ctx = TestContext::with_chain_step_no_compensation_registered().await;
    ctx.run_chain(ctx.chain_id).await;
    let r = ctx.fetch_rollback_for_step(ctx.step_id).await;
    assert_eq!(r.status, "no_compensation_registered");
}

#[tokio::test]
async fn compensation_failure_isolated() {
    let ctx = TestContext::with_3_step_chain_step2_compensation_fails().await;
    ctx.run_chain(ctx.chain_id).await;
    let rollbacks = ctx.fetch_rollbacks(ctx.chain_id).await;
    // step 2 compensation failed; step 1 still compensated
    assert_eq!(rollbacks.iter().find(|r| r.step_order == 2).unwrap().status, "compensation_failed");
    assert_eq!(rollbacks.iter().find(|r| r.step_order == 1).unwrap().status, "compensated");
}

// 5.4..5.10

§7 — Dependencies

Upstream: TASK-CUO-104. Cross-module: TASK-SKILL-001 (compensation registry), TASK-MEMORY-111 (PII).

§10 — Failure modes

FailureDetectionOutcomeRecovery
No compensation registeredlookup misspreserve + sev-3 auditmanual cleanup
Compensation execution failscatchcompensation_failed sev-2manual or retry
Cross-tenant rollbackRLS403inherent
Concurrent rollbackUNIQUE on (chain, step)second skippedinherent
Mid-rollback crashresume from last completedpartialretry
Step still running on triggerwait or abortsev-2manual
Compensation skill missinglookup failsev-2; treat as no-compensationdata fix
Rollback of skipped stepfilterno-opinherent
Trace_id missingsev-3use NIL_UUIDbug fix
Decimal precision N/Ainherentinherentinherent

§11 — Implementation notes


End of TASK-CUO-105 spec.