docs/offline-replay
Offline Replay & Benchmarks
Evaluate incident triage locally with zero external network or LLM API calls, backed by verified sub-millisecond benchmark measurements.
Zero API spend testing: The offline replay CLI runs entirely locally using bundled JSON fixtures. You can validate stack parsing, regex secret scrubbing, and deduplication without configuring GitHub or OpenAI API keys.
Running Offline Replay CLI
Execute the replay engine directly against any bundled or custom incident payload:
Terminal — Offline Replay CLI
| 1 | python -m ichnoscope replay bench/fixtures/payment_index_error.json --dry-run |
Bundled Reference Crash Fixtures
payment_index_error.jsonP1-Critical
KeyError: stripe_token
Simulates unhandled payment contract crash. Evaluates AST stack extraction and secret token scrubbing.
cart_attribute_error.jsonP2-High
AttributeError: cart_items
Simulates checkout session dictionary mismatch. Evaluates release SHA blame and 3-commit fallback.
edge_library_frame_last.jsonP3-Medium
ValueError: invalid_divisor
Simulates edge crash where vendor site-packages frames precede application code.
Measured Performance Benchmarks
Direct measurements recorded across reference fixtures:
Benchmark Latency Summary
| 1 | # Local benchmark suite results |
| 2 | # Total Fixtures: 3 | Zero Network IO | Zero LLM Tokens |
| 3 | |
| 4 | Average Core Latency: 0.09 ms per incident |
| 5 | Throughput Capacity: ~10,909 incidents/second |
| 6 | Synchronous PII Scrubbing: 0.011 - 0.022 ms |
| 7 | AST In-App Frame Extraction: 0.041 - 0.103 ms |
| 8 | Constant-Time SHA-256 Dedupe: Instant (< 0.01 ms) |