ReliBot & MyClawBot

In one breath

• ReliBot (The Hartford): incident-response agent for SRE and dev teams. Asked in Teams, CLI or web chat, it pulls Splunk logs, Dynatrace problems, recent GitHub deploys and scanner results, and answers with evidence.
• MyClawBot (personal): 12 bot instances on Telegram and WhatsApp, built on the Claude Agent SDK. The proving ground for the same memory, bridge and security patterns.
• What sets them apart: long-term memory that recalls past fixes, and a fail-closed tool policy on every call.

Key numbers

Request path

1. Channel: Teams (Bot Framework JWT/JWKS + tenant check), CLI, web chat (WebSocket).
2. Bridge: per-user rate limit → input injection scan → session rotation.
3. ADK callbacks around the root agent:
   • before_model_callback: inject L1/L2/L3 recall
   • before_tool_callback: fail-closed allowlist + call cap
   • after_tool_callback: scan tool output for injection
4. Models: Vertex AI Gemini 3 Pro → 3 Flash → 3.1 Flash → 3.1 Flash-Lite.
5. Post-turn: log the conversation, auto-save incident fixes and insights.

Recall layers (ReliBot)

• L1–L3 are pure SQL, no LLM call, injected on the first model call only.
• L3 matching is a substring / word-boundary check against a cached top-1K entity list: microseconds.

Ranking formulas

• Hybrid: 0.7 · vector + 0.3 · BM25
• Graph mode: 0.5 · hybrid + 0.3 · graph + 0.2 · entity
• MyClawBot adds: × recency decay (30-day half-life) → MMR re-rank (λ = 0.7) over a 3× over-fetch.
• Caching the embedding matrix took search from 4.6 s to 58 ms.

Write path

• During the turn: the model calls save_memory.
• Post-turn auto-save: ≥ 2 incident tools used and nothing saved → incident-resolutions/.
• Insights: answer ≥ 500 chars or > 2 turns → insights/.
• Heartbeat flush (idle 15 min or nightly): Gemini Flash pass 1 → Context · Decisions · Lessons · Actions; pass 2 → facts, SHA-1 dedup.
• Indexing on save: chunk 1,500 chars → local ONNX embedding + FTS5 triggers + regex entities (sync) + LLM entity extraction on an asyncio.Queue (async).

Storage

• ReliBot: one SQLite file in WAL mode: notes + FTS5, chunks + sqlite-vec, entities / relationships, conversations + FTS5, incidents.
• MyClawBot: a markdown vault is the source of truth (git-trackable). The SQLite index is rebuildable, and unchanged files are skipped by content hash.
• Why SQLite: each instance is team-scoped (~2.5K chunks, ~500 notes). Millisecond queries, zero ops, one file to back up.

Guardrails

• Fail-closed tool allowlist, enforced in code.
• Call caps in code: Gemini sub-agents called search_notes 15+ times and prompt instructions didn't stop them.
• Memory is an injection persistence vector: text saved once is replayed every turn. So MEMORY.md is sanitized, tool output is scanned, and compression input is fenced as quoted data.
• Isolation: one subprocess and data directory per team instance, run by a manager process.

Reliability

• Failover chain with retry backoff 1 / 3 / 8 s.
• Circuit breaker opens after 2 failures, so a bad model is skipped instead of paying its timeout on every request.
• Replies served by a fallback are tagged [fallback: model].
• Bounded growth: session rotation (summary written first), 90-day TTL, SHA-1 fact dedup.

ReliBot vs MyClawBot

Prompt-caching rule

• Caching is prefix-based: per-turn recall in the system prompt breaks the cache every turn.
• MyClawBot keys the system prompt on file mtimes and puts dynamic context in the user turn.
• Volatile content goes last.

Design decisions → at scale

Token-metering lessons

1. Cumulative counter summed as a delta. On resumed sessions the SDK's total_cost_usd is a running total. Recorded $561.97 vs $144.12 actual: 3.9× (7× on one bot).
2. Cached tokens. input_tokens excludes cache reads and writes, so rotation saw 2–8 tokens while real context passed 300K.

Rules for a metering pipeline:

• idempotent usage events (event_id), idempotency keys on billing calls
• know counter vs gauge vs delta for every provider field
• price input, output, cache-write and cache-read separately
• late or duplicate events: dedup by event_id, close periods with a grace window
• reconcile three ways: usage store vs billing system vs provider invoice

Testing

• LLM mocked at the boundary.
• Storage tests use a real SQLite in tmp_path: mocking the database hides the bugs that matter.
• Deterministic fast gate on every change, full suite before merge.