Task — engineering-spec@1

"memory put_if — optimistic-concurrency primitive with content-hash preconditions; many-agent contention without clobbering; canonical-ops extension §3.1"

doneTASK-MEMORY-118
module memory · class product · priority p1 · created 2026-05-19 · shipped null
depends on TASK-MEMORY-117 · blocks none

§1 — Description (BCP-14 normative)

The put_if operation MUST extend the canonical-op list in AGENTS.md §3.1 as an additive primitive. It is put with an explicit precondition that the existing on-disk body's SHA-256 matches a caller-supplied value. The contract:

  1. MUST add put_if to the canonical-op enum in memory.schema.json. Signature: put_if(path, body, meta, precondition_body_hash: Optional[str]) -> PutIfResult. The two-arg path/body shape mirrors put; the new third arg is the precondition.
  2. MUST define precondition_body_hash as either:
  1. MUST acquire the same .lock (exclusive) that put does. Inside the lock:
  1. MUST NOT advance HEAD on rejection. The audit chain remains unchanged. A memory.precondition_failed aux audit row IS emitted (HEAD advances by exactly 1 for the aux row); see §1 #7. The put payload itself is never written.
  2. MUST honour the TASK-MEMORY-117 store ACL check IN ADDITION to the precondition check. ACL check runs FIRST: if ACL rejects, return rejected with reason: "acl_denied". Only if ACL allows does the precondition check fire. Rationale: ACL is policy; precondition is concurrency control; policy must clear before concurrency control matters.
  3. MUST preserve all existing put semantics on success: same canonical row emission, same extra.body_hash annotation, same extra field passthrough. From the audit chain's perspective, a successful put_if is indistinguishable from a put — only the writer's pre-step differs.
  4. MUST emit memory.precondition_failed aux audit row on rejection with payload: ``json { "actor": "stephen", "path": "memories/sre/dispatch-1.md", "expected": "abc123…" | null, "actual": "def456…" | "<absent>", "attempt_at": "2026-05-19T08:00:00Z" } ``
  5. MUST add a CLI surface cyberos put-if <path> <body-source> --precondition <hex|none>. --precondition none is the null-precondition form. --precondition-from-file <path> reads the hex from a file (for scripted operators).
  6. MUST return the structured result type: ``python @dataclass class PutIfResult: outcome: Literal["written", "rejected"] reason: Optional[str] # populated when outcome == "rejected" expected: Optional[str] actual: Optional[str] committed_seq: Optional[int] # the HEAD seq of the put row on success ``
  7. MUST validate precondition_body_hash shape at API entry: 64-char lowercase hex OR None. Other shapes ("abc", byte strings, hash with sha256: prefix) raise ValueError.
  8. MUST be safe under retry loops — caller pattern: ``python while True: current = reader.read(path) new_body = update(current.body) res = writer.put_if(path, new_body, current.meta, precondition_body_hash=current.body_hash) if res.outcome == "written": break # res.outcome == "rejected" — re-read and retry `` The protocol provides the primitive; the retry policy is the caller's responsibility. Document the canonical pattern in §11.
  9. MUST require the AGENTS.md §3.1 amendment APPROVED via APPROVE protocol change P21 §3.1 chat-turn before put_if is callable. Same anchor-check pattern as TASK-MEMORY-115 / 117: Writer.put_if(...) raises ProtocolAmendmentMissingError if §3.1 hasn't been extended to include put_if.
  10. SHOULD be implementable in INTEROP.md consumers — the precondition check is one-line (compute SHA-256 of the file body before write). Cross-agent interop note documented in INTEROP.md.
  11. SHOULD support a batch variant put_if_all([(path, body, meta, precond), ...]) that succeeds-all or rejects-all transactionally. Slice-4 stretch — useful for TASK-MEMORY-115 dream-apply (which already has its own transactional path).

§2 — Why this design (rationale for humans)

Why additive, not replacement (§1 #1, DEC-240). The simple put is the right primitive for most operators (script writes one memory, doesn't care about race). Forcing every caller through precondition would punish the common case. put_if is the explicit choice when concurrency matters.

Why SHA-256 body_hash as precondition (§1 #2, DEC-241). Two reasons. (a) Content-equality is the natural primitive for memory files — two files with the same content are semantically the same. (b) meta.body_hash is already computed + stored by the writer for every memory; reusing it costs zero extra hashing. The caller can read the current file's frontmatter, grab body_hash, pass it back.

Why null precondition for create-only (§1 #2). Common pattern: "create this memory only if it doesn't exist." Without a null option, callers would compute the absent-file's "hash" as some magic value, which is awkward. null is unambiguous.

Why ACL check first (§1 #5). Two reasons. (a) Policy enforcement is more important than concurrency control; deny-first is safer. (b) An ACL-denied write should report acl_denied, not precondition_failed — different operator action required. Running ACL first makes the error specific.

Why aux row even on rejection (§1 #4, §1 #7, DEC-242). Dream-applier (TASK-MEMORY-115) and FS watcher (TASK-MEMORY-107) need to count rejections to detect contention storms. Operators investigating "why did agent X's writes silently disappear?" need a queryable log. The aux row is cheap; visibility is high-value.

Why preserve put semantics on success (§1 #6). put_if shouldn't pollute the audit chain with a new row shape — downstream consumers (walker, doctor, TASK-MEMORY-115's pattern detector, TASK-MEMORY-120's history view) all already understand put rows. Making put_if's success row identical to a put row keeps every consumer working without changes.

Why §3.1 amendment required (§1 #12, DEC-242). §3.1 currently lists exactly three canonical ops (put, move, delete). Adding put_if is a normative extension to that list; INTEROP.md consumers need to know it exists. That's a protocol change, gated by APPROVE per §0.2.

Why batch variant deferred (§1 #14). Dream-applier already has its own transactional path. A general-purpose batch primitive needs a careful design (rollback semantics, partial-failure reporting, lock-window length). Slice-4 work.


§3 — API contract

Method signature

# modules/memory/cyberos/core/writer.py — extension
from dataclasses import dataclass
from typing import Literal, Optional


@dataclass(frozen=True)
class PutIfResult:
    outcome:       Literal["written", "rejected"]
    reason:        Optional[str] = None
    expected:      Optional[str] = None
    actual:        Optional[str] = None
    committed_seq: Optional[int] = None


class Writer:
    # ... existing methods ...

    def put_if(
        self,
        path: str,
        body: bytes | str,
        meta: dict,
        precondition_body_hash: Optional[str] = None,
    ) -> PutIfResult:
        self._require_protocol_amendment_p21()                  # §1 #12

        # Shape check (§1 #10)
        if precondition_body_hash is not None:
            if not (isinstance(precondition_body_hash, str)
                    and len(precondition_body_hash) == 64
                    and all(c in "0123456789abcdef" for c in precondition_body_hash)):
                raise ValueError(
                    f"precondition_body_hash must be 64-char lowercase hex or None; "
                    f"got {precondition_body_hash!r}"
                )

        # ACL check (§1 #5) — runs before lock to fail fast
        acl_result = self._check_acl_write(path)
        if not acl_result.allowed:
            self._emit_aux("memory.acl_denied", {"path": path, **acl_result.payload_dict()})
            return PutIfResult(outcome="rejected", reason="acl_denied")

        with self._exclusive_lock():
            # Precondition check (§1 #3)
            existing_path = self.store_path / path
            existing_body = existing_path.read_bytes() if existing_path.exists() else None
            existing_hash = (
                hashlib.sha256(existing_body).hexdigest() if existing_body else None
            )

            match precondition_body_hash, existing_hash:
                case None, None:
                    pass                            # create-only: target absent ✓
                case None, _:
                    self._emit_aux("memory.precondition_failed", {
                        "path": path, "expected": None, "actual": existing_hash,
                        "attempt_at": _now_iso(),
                    })
                    return PutIfResult(outcome="rejected", reason="precondition_failed",
                                       expected=None, actual=existing_hash)
                case _, None:
                    self._emit_aux("memory.precondition_failed", {
                        "path": path, "expected": precondition_body_hash,
                        "actual": "<absent>", "attempt_at": _now_iso(),
                    })
                    return PutIfResult(outcome="rejected", reason="precondition_failed",
                                       expected=precondition_body_hash, actual="<absent>")
                case provided, actual if provided != actual:
                    self._emit_aux("memory.precondition_failed", {
                        "path": path, "expected": provided,
                        "actual": actual, "attempt_at": _now_iso(),
                    })
                    return PutIfResult(outcome="rejected", reason="precondition_failed",
                                       expected=provided, actual=actual)
                case _:
                    pass                            # match ✓

            # Proceed with the same put logic as the existing `put(path, body, meta)`
            committed_seq = self._put_internal(path, body, meta)
            return PutIfResult(outcome="written", committed_seq=committed_seq)

AGENTS.md §3.1 amendment (proposed)

## §3.1 (extended by P21 — requires APPROVE chat-turn per §0.2)

An agent operating on memory state MUST express every mutation as exactly
one of FOUR canonical operations:

| op       | semantic |
|----------|----------|
| `put`    | create or replace a memory file. Idempotent given identical args. |
| `move`   | rename within `<memory-root>/`. Preserves content hash. |
| `delete` | `mode ∈ {"tombstone", "purge"}`; default `"tombstone"`. |
| `put_if` | create or replace, GATED on `precondition_body_hash` (SHA-256 of  |
|          | current body) matching. `None` precondition ≡ "must not exist".   |

§3.1.5  `put_if` MUST emit `memory.precondition_failed` aux audit row on
mismatch; HEAD does NOT advance for the rejected `put` payload but DOES
advance by 1 for the aux row.

§3.1.6  `put_if` is INDISTINGUISHABLE from `put` in the success-row shape.
Downstream consumers (walker, doctor, dream pipeline) MUST NOT special-case
`put_if`-origin `put` rows.

§4 — Acceptance criteria

  1. Precondition match → write succeeds — read existing file, hash, call put_if with that hash → outcome written; HEAD advances by 1 (put row). (traces_to: §1 #3)
  2. Precondition mismatch → write rejected — pass a different hash → outcome rejected; reason precondition_failed; expected + actual fields populated; HEAD advances by 1 (aux row only). (traces_to: §1 #3, §1 #4)
  3. Null precondition + path absent → write succeeds — target doesn't exist, precondition=null → outcome written. (traces_to: §1 #2, §1 #3)
  4. Null precondition + path present → rejected — target exists, precondition=null → outcome rejected; expected=null, actual=<current hash>. (traces_to: §1 #2, §1 #3)
  5. Hash + path absent → rejected — target doesn't exist, precondition=<some hash> → outcome rejected; expected=<provided>, actual="<absent>". (traces_to: §1 #3)
  6. Two concurrent put_if race → one wins — simulate two threads with the same precondition; one returns written, the other returns rejected. (traces_to: §1 #3)
  7. HEAD doesn't advance on rejection (put row not emitted) — only the aux row is emitted, not a put row. Inspect HEAD seq before + after rejected call. (traces_to: §1 #4)
  8. ACL denial reports acl_denied, not precondition_failed — STORE.yaml denies; valid precondition supplied → outcome rejected with reason acl_denied. (traces_to: §1 #5)
  9. ACL check runs before precondition — instrument both; deny-by-acl + wrong-precondition → only ACL check fires; no memory.precondition_failed row. (traces_to: §1 #5)
  10. Success row indistinguishable from put — successful put_if emits same row shape as direct put (same extra keys, same canonical fields). (traces_to: §1 #6)
  11. Aux row payload shapememory.precondition_failed row matches §1 #7 schema (actor, path, expected, actual, attempt_at). (traces_to: §1 #7)
  12. CLI put-if --precondition <hex> works — happy path. (traces_to: §1 #8)
  13. CLI put-if --precondition none for create-only — alias for null. (traces_to: §1 #8)
  14. CLI put-if --precondition-from-file — reads hex from file. (traces_to: §1 #8)
  15. PutIfResult shape — typed result has all fields (outcome, reason, expected, actual, committed_seq). (traces_to: §1 #9)
  16. Bad precondition shape rejectedput_if(..., precondition_body_hash="abc") raises ValueError; precondition_body_hash=b"\xab\xcd…" (bytes) raises. (traces_to: §1 #10)
  17. Bad precondition with uppercase rejected"ABC123..." (uppercase hex) raises (must be lowercase per spec). (traces_to: §1 #10)
  18. Retry-loop pattern works end-to-end — write a memory; concurrently the file gets overwritten; retry loop re-reads + retries; eventual written. (traces_to: §1 #11)
  19. §3.1 anchor required — AGENTS.md §3.1 missing the put_if extension → Writer.put_if(...) raises ProtocolAmendmentMissingError. (traces_to: §1 #12)
  20. §3.1 anchor present → put_if callable — AGENTS.md amended with put_if row in the canonical-op table → callable. (traces_to: §1 #12)

§5 — Verification

# modules/memory/tests/test_put_if_optimistic_concurrency.py
import hashlib, threading, pytest
from cyberos.core.writer import Writer, PutIfResult


def hash_body(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()


def test_precondition_match_writes(seeded_memory_with_x, ensure_section_3_1):
    """AC #1"""
    current = (seeded_memory_with_x.store_path / "memories/x.md").read_text()
    h = hash_body(current)
    head_before = seeded_memory_with_x.head_seq()
    res = seeded_memory_with_x.put_if("memories/x.md", current + "\nappended",
                                      meta={}, precondition_body_hash=h)
    assert res.outcome == "written"
    assert res.committed_seq == head_before + 1


def test_precondition_mismatch_rejects(seeded_memory_with_x, ensure_section_3_1):
    """AC #2 + #7"""
    head_before = seeded_memory_with_x.head_seq()
    res = seeded_memory_with_x.put_if("memories/x.md", "new body", meta={},
                                      precondition_body_hash="0" * 64)
    assert res.outcome == "rejected"
    assert res.reason == "precondition_failed"
    # HEAD advances by 1 (aux row), not 2 (no put row)
    assert seeded_memory_with_x.head_seq() == head_before + 1


def test_null_precondition_path_absent_writes(empty_memory, ensure_section_3_1):
    """AC #3"""
    res = empty_memory.put_if("memories/new.md", "fresh body", meta={},
                              precondition_body_hash=None)
    assert res.outcome == "written"


def test_null_precondition_path_present_rejects(seeded_memory_with_x, ensure_section_3_1):
    """AC #4"""
    res = seeded_memory_with_x.put_if("memories/x.md", "trying to create over existing",
                                      meta={}, precondition_body_hash=None)
    assert res.outcome == "rejected"
    assert res.expected is None
    assert res.actual is not None


def test_hash_precondition_path_absent_rejects(empty_memory, ensure_section_3_1):
    """AC #5"""
    res = empty_memory.put_if("memories/absent.md", "body", meta={},
                              precondition_body_hash="a" * 64)
    assert res.outcome == "rejected"
    assert res.actual == "<absent>"


def test_concurrent_put_if_one_wins(seeded_memory_with_x, ensure_section_3_1):
    """AC #6"""
    current = (seeded_memory_with_x.store_path / "memories/x.md").read_text()
    h = hash_body(current)
    results = []
    barrier = threading.Barrier(2)

    def attempt(suffix):
        barrier.wait()
        res = seeded_memory_with_x.put_if("memories/x.md", current + suffix,
                                          meta={}, precondition_body_hash=h)
        results.append(res)

    t1 = threading.Thread(target=attempt, args=("\nfrom-1",))
    t2 = threading.Thread(target=attempt, args=("\nfrom-2",))
    t1.start(); t2.start(); t1.join(); t2.join()

    outcomes = sorted(r.outcome for r in results)
    assert outcomes == ["rejected", "written"]


def test_acl_denial_reported_specifically(seeded_memory_with_x, deny_acl,
                                          ensure_section_3_1):
    """AC #8 + #9"""
    h = hash_body((seeded_memory_with_x.store_path / "memories/x.md").read_text())
    res = seeded_memory_with_x.put_if("memories/x.md", "new", meta={},
                                      precondition_body_hash=h)
    assert res.outcome == "rejected"
    assert res.reason == "acl_denied"
    # No precondition_failed row emitted
    rows = seeded_memory_with_x.read_recent_audit_rows(2)
    assert not any(r["kind"] == "memory.precondition_failed" for r in rows)


def test_success_row_indistinguishable_from_put(seeded_memory_with_x, ensure_section_3_1):
    """AC #10"""
    h = hash_body((seeded_memory_with_x.store_path / "memories/x.md").read_text())
    res = seeded_memory_with_x.put_if("memories/x.md", "new body", meta={},
                                      precondition_body_hash=h)
    put_row = seeded_memory_with_x.read_audit_row(res.committed_seq)
    assert put_row["kind"] == "put"           # NOT "put_if"
    # Shape matches the existing put row exactly
    assert set(put_row.keys()) >= {"kind", "payload", "extra"}


def test_aux_row_payload_shape(seeded_memory_with_x, ensure_section_3_1):
    """AC #11"""
    head_before = seeded_memory_with_x.head_seq()
    seeded_memory_with_x.put_if("memories/x.md", "new", meta={},
                                precondition_body_hash="0" * 64)
    aux = seeded_memory_with_x.read_audit_row(head_before + 1)
    assert aux["kind"] == "memory.precondition_failed"
    for key in ("path", "expected", "actual", "attempt_at"):
        assert key in aux["payload"]


@pytest.mark.parametrize("bad", [
    "abc",                            # too short
    "x" * 64,                          # non-hex
    "ABC" + "0" * 61,                  # uppercase (per spec lowercase only)
    b"\xab" * 32,                       # bytes
    123,                                # int
])
def test_bad_precondition_shape_rejected(empty_memory, ensure_section_3_1, bad):
    """AC #16 + #17"""
    with pytest.raises(ValueError):
        empty_memory.put_if("memories/x.md", "body", meta={},
                            precondition_body_hash=bad)


def test_protocol_amendment_required(seeded_memory_without_section_3_1):
    """AC #19"""
    with pytest.raises(Exception) as exc:
        seeded_memory_without_section_3_1.put_if("memories/x.md", "x", meta={},
                                                  precondition_body_hash=None)
    assert "APPROVE protocol change P21 §3.1" in str(exc.value)


def test_retry_loop_eventual_write(seeded_memory_with_x, ensure_section_3_1,
                                    background_overwriter):
    """AC #18"""
    final = None
    for _ in range(5):
        current = (seeded_memory_with_x.store_path / "memories/x.md").read_text()
        h = hash_body(current)
        # Concurrent overwriter just touched the file again; retry might fail
        res = seeded_memory_with_x.put_if("memories/x.md", current + "\nfinal",
                                          meta={}, precondition_body_hash=h)
        if res.outcome == "written":
            final = res; break
    assert final is not None and final.outcome == "written"


def test_cli_put_if_works(seeded_memory_with_x, ensure_section_3_1, capsys):
    """AC #12"""
    import subprocess
    h = hash_body((seeded_memory_with_x.store_path / "memories/x.md").read_text())
    result = subprocess.run([
        "python", "-m", "cyberos", "--store", str(seeded_memory_with_x.store_path),
        "put-if", "memories/x.md", "-", "--precondition", h
    ], input="updated body", capture_output=True, text=True)
    assert result.returncode == 0
    assert "written" in result.stdout


def test_cli_put_if_none(seeded_memory_with_x, ensure_section_3_1):
    """AC #13"""
    import subprocess
    result = subprocess.run([
        "python", "-m", "cyberos", "--store", str(seeded_memory_with_x.store_path),
        "put-if", "memories/new.md", "-", "--precondition", "none"
    ], input="fresh body", capture_output=True, text=True)
    assert result.returncode == 0
    assert "written" in result.stdout

§6 — Implementation skeleton

API + tests above are the skeleton. Order:

  1. AGENTS.md §3.1 amendment text (DO NOT commit until APPROVE chat-turn).
  2. Schema (memory.schema.json) — add put_if op.
  3. Walker invariant put-if-precondition-form.
  4. Writer extension: put_if method + result type.
  5. CLI subcommand.
  6. Tests.
  7. INTEROP.md note.
  8. CHANGELOG.

§7 — Dependencies


§8 — Example payloads

Successful put_if

$ cyberos put-if memories/sre/dispatch-1.md - --precondition abc123def456789a...
< updated body content
Result: {"outcome":"written","committed_seq":4319}

Rejected on mismatch

$ cyberos put-if memories/sre/dispatch-1.md - --precondition 0000000000000000...
< new body
Result: {"outcome":"rejected","reason":"precondition_failed","expected":"00000000...","actual":"abc123de..."}

Rejected on ACL

$ cyberos put-if memories/org-wide-knowledge/runbook.md - --precondition <hash> --actor scheduled-importer
Result: {"outcome":"rejected","reason":"acl_denied"}

memory.precondition_failed aux row

{
  "kind": "memory.precondition_failed",
  "payload": {
    "actor":       "stephen",
    "path":        "memories/sre/dispatch-1.md",
    "expected":    "0000000000000000000000000000000000000000000000000000000000000000",
    "actual":      "abc123def456789abcdef0123456789abcdef0123456789abcdef0123456789a",
    "attempt_at":  "2026-05-19T08:00:00Z"
  }
}

§9 — Open questions

All resolved. Deferred:


§10 — Failure modes inventory

FailureDetectionOutcomeRecovery
§3.1 missing put_if_require_protocol_amendment_p21raises ProtocolAmendmentMissingErrorOperator runs APPROVE chat-turn
Precondition shape invalidshape check at API entryValueErrorCaller fixes hash format
ACL deniescheck before preconditionrejected w/ reason=acl_deniedCaller checks ACL or uses different actor
Two concurrent put_if same preconditionlock serialisesone written, other rejectedCaller retries
File deleted between read and put_ifprecondition has hash, actual=<absent>rejectedCaller decides: re-read + null-precondition put_if, or give up
File renamed between read and put_ifpath doesn't existrejected with actual=<absent>Caller handles
Bytes input vs str inputwriter's normal handlingworksNone
Empty bodyaccepted; hash of empty string is well-definedworksNone
Very large bodynormal put performanceworksNone
Concurrent put (not put_if) clobbersput doesn't check preconditionslast writer wins (existing semantics)Caller uses put_if instead of put
Concurrent read between two put_ifsreader sees consistent state at any pointNone - by designNone
Hash collision (cosmic ray)cryptographically improbablen/an/a
HEAD advances by 0 on rejectionaux row IS emittedHEAD +1None — aux row is the audit
Caller passes uppercase hexshape checkValueErrorCaller lowercases
Memory file is encrypted (task §5.4)precondition is computed on the cipher BYTES (not plaintext)works because both sides see the same ciphertextNone
File has UTF-8 BOMhash computed on raw bytesworks (BOM is part of body)None
Caller forgets --precondition flagargparse required argCLI errorCaller adds flag

§11 — Implementation notes


End of TASK-MEMORY-118.

As built (2026-07-02)

Put-if-precondition logic shipped inside modules/memory.