Memory Replay For Forensics: Reconstructing What An Agent Saw On The Day It Failed

TL;DR

When an agent fails, the only path to root cause is reconstruction: what did the agent see, what model produced the response, what prompt and tool schema were active, what reasoning trace led to the action. This is forensic replay, and it requires capturing more than the raw decision. The capture spec includes the rendered context window byte-for-byte, all retrieved memory entries with their tier and provenance, the exact system prompt version, the tool schema in effect, the model identifier and version, the temperature and other sampling parameters, the tool outputs the agent received, and the trace of intermediate reasoning. Without this, your post-mortem is anecdotal. With it, you can replay the failure deterministically, isolate the variable that caused it, and ship a fix you can defend. Replay is the line between agent operations as engineering and agent operations as folklore.

The Anatomy Of A Failure You Cannot Investigate

An agent fails in production. The customer complains. Your team gets paged. You open the agent's logs and see a clean record of what the agent said and did: the user message at 14:32, the agent response at 14:32:08, a tool call at 14:32:09, an outcome at 14:32:11. Everything is structured, timestamped, and useless. You can see what happened. You cannot see why it happened.

Why is the part the logs do not capture. The agent retrieved some memory before generating the response. Which memory? The logs do not say. The agent had some system prompt in effect. Which version? The logs do not say. The agent's response shows reasoning patterns that suggest a particular fact was salient. Was that fact actually in context, or did the model hallucinate it? The logs do not say. Your team starts speculating: "It probably retrieved the wrong customer record because we shipped that retrieval change last week." Maybe. You cannot prove it. You cannot reproduce it. You cannot ship a fix you can defend, because you do not know what you are fixing.

The pattern continues. You patch something that seems related, redeploy, and watch nervously to see if the failure recurs. If it does not recur, you assume the patch worked, but you cannot be sure. If it does recur, you patch something else. Over weeks, the agent's reliability creeps in the wrong direction, and your team's confidence creeps with it. Eventually you start saying things like "the agent is acting weird," because you have lost the ability to say specifically what is wrong.

This is the state of agent operations without forensic replay. It is the state of an engineering practice that has not yet been forced to accept that agents need the same observability disciplines as any other production system, and then some, because the variables that affect agent behavior are richer and more numerous than the variables that affect a typical service. The fix is replay: capture enough at decision time that you can reconstruct the entire input space the agent was operating over, and re-execute the decision under controlled conditions to isolate the cause.

What Replay Actually Means

Replay means re-executing a past decision with byte-identical inputs to see what the agent does today, then varying inputs systematically to identify which variable is responsible for the difference between the failure and the desired outcome. The first part requires that you captured everything; the second part requires that you can edit one variable at a time and re-run.

Byte-identical inputs is the load-bearing constraint. If you replay with "the same" memory entries but the entries have been compressed since, you are not replaying; you are running a similar-looking new decision. If you replay with "the same" system prompt but the prompt template has been edited since, you are running a different agent. If you replay with "the same" model but the model has been updated to a new version, you are running a different model. The temptation to skip these constraints because they are operationally inconvenient is strong. Yielding to the temptation produces non-replay: you reproduce the situation but not the failure, and you draw confident conclusions from a session that did not actually reproduce what you were investigating.

The second part (variable isolation) is the part that produces root cause. Once you can replay the failure faithfully, you can start changing things. Replace the retrieved memory with a different set: does the failure still happen? If yes, the failure is not in retrieval. If no, the failure is in retrieval. Replace the system prompt with the prior version: does the failure still happen? If yes, the prompt change is not the cause. If no, the prompt is the cause. This bisection-by-variable is how you go from "something is wrong" to "this specific change introduced this specific behavior." Each variable you can hold constant is a variable you can clear. Each variable you can vary independently is a variable you can attribute.

The practical workflow is: capture a failure (ideally automatically when the failure indicator fires); replay it under current conditions to confirm the failure reproduces; replay under prior conditions to identify when the failure was introduced (this is where memory state, prompt versions, and model versions matter); bisect across the changes between then and now to isolate the responsible change; ship a fix; replay again to verify the fix resolves the failure without introducing new ones. This is the same loop you use for any production debugging, with one critical addition: the inputs are not just request parameters. They are an entire memory and prompt and model state that you have to be able to reconstruct.

Reader Artifact: The Forensic Replay Capture Specification

The specification below enumerates everything that must be captured at decision time to enable byte-faithful replay. It is intentionally conservative: capture more than you think you need, because you cannot reconstruct what you did not capture.

{
  "version": "replay/v1",
  "decisionId": "dec_...",
  "agentId": "agt_...",
  "runtimeId": "rt_...",
  "capturedAt": "2026-07-23T13:00:00.456Z",

  "input": {
    "originUserId": "...",
    "originSessionId": "...",
    "requestPayload": {... },         // the inbound request that triggered the decision
    "requestPayloadHash": "sha256:..."
  },

  "agentConfiguration": {
    "systemPromptVersion": "v4.7",
    "systemPromptHash": "sha256:...",
    "systemPromptText": "...",          // verbatim text in effect
    "toolSchemaVersion": "...",
    "toolSchemaHash": "sha256:...",
    "toolSchemaJson": {... },          // verbatim schema available to the model
    "modelIdentifier": "...",
    "modelVersion": "...",
    "modelProviderEndpoint": "...",
    "samplingParameters": {
      "temperature": 0.2,
      "top_p": 0.95,
      "max_tokens": 4096,
      "stop": [...]
    }
  },

  "memoryState": {
    "workingSetEntries": [
      {
        "entryId": "mem_...",
        "tier": "hot",
        "provenance": { "writerId": "...", "writtenAt": "...", "sourceType": "..." },
        "renderedContent": "...",        // exactly as inserted into context
        "renderedContentHash": "sha256:...",
        "contextPosition": 3,
        "retrievalScore": 0.84,
        "retrievalMethod": "semantic | bm25 | tag_filter",
        "embeddingModel": "...",
        "embeddingVersion": "..."
      }
    ],
    "totalContextTokens": 12483,
    "contextSetHash": "sha256:..."      // Merkle root over working set
  },

  "executionTrace": {
    "reasoningTokens": [...],            // if the model produced visible reasoning
    "reasoningTokenCount": 1240,
    "toolCalls": [
      {
        "sequence": 1,
        "toolName": "...",
        "input": {... },
        "output": {... },
        "outputHash": "sha256:...",
        "latencyMs": 142,
        "errorRef": null
      }
    ],
    "finalOutputText": "...",
    "finalOutputHash": "sha256:...",
    "stopReason": "end_turn | max_tokens | tool_call | error",
    "totalLatencyMs": 1840
  },

  "externalState": {
    "observableExternals": [
      {
        "key": "customer.balance",
        "capturedValue": "...",
        "capturedAt": "...",
        "sourceEndpoint": "..."
      }
    ]
  },

  "replayPolicy": {
    "retentionDays": 365,
    "piiRedacted": false,
    "replayAccessScope": "operator | regulator | counterparty"
  },

  "signature": {
    "algorithm": "ed25519",
    "publicKeyRef": "did:armalo:agent:...#runtime-key-1",
    "signatureBytes": "base64:..."
  }
}

The specification is bigger than people expect. That is the point. Replay needs the whole picture. Captures that omit the rendered memory content are useless because retrieval cannot be faithfully reconstructed from a present-day query. Captures that omit the system prompt are useless because the agent's behavioral envelope is encoded there. Captures that omit the tool schema are useless because the model's available actions are constrained by it. Captures that omit the model version are useless because models drift; the same identifier today may not be the same weights as a year ago.

The overhead is non-trivial: a typical capture is tens of kilobytes, sometimes hundreds depending on context size and tool output volume. Storage cost is real. Storage cost is also tiny compared to the cost of failing to investigate a production incident. Capture every decision that has external consequence; capture sample fractions of routine turns to give yourself replay fodder for behavioral drift analysis. Do not capture nothing.

When To Replay, And What To Replay With

Replay is invoked under three conditions. The first is incident response: a specific failure has been reported, you have the decision ID, and you want to understand what happened. Replay this single decision. Vary one variable at a time. Find the cause.

The second is regression detection: you have shipped a change (a prompt update, a model version bump, a memory architecture migration) and you want to verify that historical decisions would still behave correctly under the new conditions. Replay a sample of past decisions under the new configuration. Compare outcomes to the captured originals. Failures in the comparison are regressions you introduced; investigate them before the change reaches the failure-prone tail of users.

The third is adversarial probing: a security researcher reports that a particular pattern of input causes problematic behavior. Replay decisions with adversarial variations to map the boundary of the failure mode. This is what red-team work looks like in production: not synthetic probes against a clean instance, but probes against captured decisions to find which kinds of agents fail in which kinds of ways.

The replay environment matters. The wrong way to replay is in production. Replaying a failed decision in production may re-execute the failed action, which may have already-bad consequences and adding a duplicate makes them worse. The right way is in a sandbox that uses the captured external state instead of calling the real external systems. Tool calls in replay get simulated against the captured outputs from the original execution; they do not actually mutate state. This requires the tool schema to be replay-aware and the runtime to be able to substitute a recorded-output mode for live-execution mode. It is engineering work. It is essential engineering work.

The captured external state is the part of the spec that newcomers underestimate. An agent's decision depends not only on its memory but on the live state of the world it queried. A customer's account balance, a market price, a third-party API's response, an external service's availability: all of these affect the decision and all of them change between decision time and replay time. Capturing the observable externals at decision time means the replay can reconstruct the world the agent saw, not the world that exists today. Without this, you are replaying against a different world and your variable isolation is contaminated.

Replay And The Memory-Trust Coupling

Replay interacts with the memory-trust coupling formula in two ways. First, replay captures count as audit-only memory access; they read but do not promote, and they do not affect score or coupling. This is the appropriate treatment because forensic access is exactly what cold-tier replay was designed for and penalizing it would discourage the behavior we want.

Second, replay can identify cases where the memory state at decision time turned out to be flawed (the agent retrieved a stale entry, the entry contained a confabulated fact, the entry was correct but the agent misused it). These findings feed back into the agent's behavioral record: if replay establishes that a class of failures shares a memory pattern, that pattern can be encoded as a future safety check, and the agent's score reflects the pattern's incidence. Replay is not just for diagnosing single failures; it is for accumulating evidence about systematic behaviors that need to feed back into governance.

The loop closes when replay-driven findings inform pact updates. "This agent has a history of failing when retrieved memory exceeds 8KB and the question involves time-bounded commitments." That is a behavioral pattern worth encoding into the agent's pact: "Agent will reduce context to 4KB on time-bounded commitment questions and surface uncertainty if reduction is not possible." The pact change is itself a versioned event; the next round of replay can verify the change had the intended effect. This is what an engineering loop on agent behavior looks like, and it does not exist without forensic replay capability.

Counter-Argument: This Is Too Expensive To Capture For Every Decision

The honest counter-argument is that capturing this much per decision is prohibitive at scale. An agent doing a thousand turns a minute generating a hundred kilobytes of capture per decision is producing six gigabytes of capture per minute. Multiplied across a fleet, the storage and processing costs are real.

The answer is selective capture, not zero capture. Three levels: full capture for any decision flagged as consequential (matches the attestation policy roughly), structured-only capture for routine turns (tool calls, outcomes, retrieval IDs but not rendered content), and sampling for the bulk of low-stakes turns (full capture for one in a hundred or one in a thousand, sampled randomly for behavioral baseline analysis). The selective approach captures the decisions you actually want to investigate at full fidelity while keeping aggregate storage bounded.

The second answer is short-window full capture: hold the full capture for 30 days against every decision, then archive everything except the consequential decisions to compact metadata-only records. This means recent failures (the ones most likely to need investigation) have full replay capability while older failures have only metadata, which is enough for trend analysis but not for byte-faithful replay. Most incidents get reported within days of occurrence, so the 30-day window covers the high-value zone.

The third answer is that storage is cheap and engineer time spent guessing at root cause is expensive. The economic comparison usually favors capturing more than you think you need. A team that can resolve incidents in hours instead of days because they have replay data is worth more than the storage bill.

What Armalo Does

The Cortex memory subsystem produces forensic replay captures for every decision flagged as consequential by the agent's pact, and structured-only captures for routine turns. Captures include the full rendered context, system prompt, tool schema, model identifier, sampling parameters, retrieved memory entries with provenance and tier, execution trace including tool calls and reasoning tokens, observable external state at decision time, and signature against the runtime working key. Replays run in a sandboxed environment that substitutes captured external state for live calls, ensuring replay does not mutate production state. The replay environment can vary one configuration variable at a time (prompt version, model identifier, retrieval set, tool schema) for systematic root-cause investigation. Replay-driven findings can be fed back into the agent's pact as updated behavioral commitments, with the pact change automatically reflected in the trust oracle. Capture access is governed: operators can replay their own agents' captures by default, regulators and counterparties can request access through the audit endpoint with appropriate authority, and all access is logged.

FAQ

Q: How is replay different from logging? A: Logging records what happened. Replay records enough to re-execute what happened. The difference is the reconstructibility: logs tell you the agent retrieved memory; replay captures tell you exactly which entries, in what order, with what content, so you can hand them to the model again.

Q: Does replay work for non-deterministic models? A: Yes, with caveats. The capture preserves the sampling parameters; replay with temperature 0 and identical inputs typically produces identical outputs from the same model version. Models that have non-determinism even at temperature 0 (some recent reasoning models exhibit this) require multiple replay runs to characterize the variance, but the variance itself is informative.

Q: What happens when the model version is no longer available for replay? A: This is a real problem. Mitigations include (a) negotiating model version retention with your provider, (b) maintaining local mirrors of model versions for critical replay needs, or (c) accepting that very old captures may be replayable only against newer model versions, with the caveat noted in the replay output.

Q: How do you replay decisions that involved real-time external data? A: The capture includes the observable external state the agent saw. Replay substitutes that captured state instead of calling the live external system. This means replay reconstructs the agent's view of the world, not today's world.

Q: Can replays affect production? A: Not by default. Replays run in sandbox mode where tool calls are simulated against captured outputs. There is an explicit "live replay" mode for cases where you want to see what would happen against current external state, but this mode requires explicit operator authorization because it can mutate state.

Q: What about replays for compliance or legal purposes? A: Replay captures are evidentiary artifacts: signed, timestamped, and reconstructible. They are admissible in dispute resolution and regulatory inquiries with the same standing as attestation manifests, and in fact they typically include the attestation manifest as a subset.

Q: How do you protect captures from tampering? A: Captures are signed by the runtime working key with a timestamp from a trusted clock source. The signing chain is the same as for attestation manifests and resolves through public identity records, so a tampered capture would fail signature verification.

Q: What is the relationship between replay and reproducible builds? A: Conceptually similar: both are about being able to reconstruct an artifact from its inputs. Reproducible builds reconstruct binaries from source; replay reconstructs agent decisions from captured state. The discipline of capturing all inputs is the load-bearing similarity.

Bottom Line

Agent failures without replay capability are folklore. You speculate, you patch, you redeploy, you hope. Agent failures with replay capability are engineering. You capture, you reproduce, you bisect, you isolate, you fix, you verify. The difference is the ability to reconstruct exactly what the agent saw at the moment it acted. This requires capturing more than logs do: the rendered context, the prompt version, the tool schema, the model identifier, the sampling parameters, the retrieved memory entries with provenance, the execution trace, and the observable external state at decision time. Build the capture spec, run the captures, build the sandbox replay environment, and your team can resolve incidents in hours instead of days. Skip it and your agent operations practice stays in the speculative-debugging era forever. The capture is the precondition for everything else.