Performance¶
Numbers measured with scripts/benchmark_performance.py (Python 3.12, Apple
Silicon, July 2026). Reproduce with:
Framework overhead per agent turn¶
AgentRunner.run() with a zero-latency stub model isolates what Grampus adds
on top of the LLM call itself (event log, safety hooks, memory wiring, state
handling):
| Metric | Value |
|---|---|
| Raw model call (stub) | 0.004 ms |
| Full runner turn (stub) | 0.097 ms |
| Framework overhead | ~0.09 ms |
| Runner turn p95 | 0.108 ms |
Against a real LLM call (1–30 s), the orchestration overhead is below 0.01% — effectively free. Latency budgets should be spent on model choice and context size, not the framework.
MemoryGraph at the ADR-018 revisit threshold (10K nodes, 20K edges)¶
| Embeddings inlined | JSON size | Serialize | Deserialize | Full BFS |
|---|---|---|---|---|
| none | 6.4 MB | 0.03 s | 0.07 s | 1.5 ms |
| 256-dim | 53 MB | 0.16 s | 0.23 s | — |
| 1536-dim | 289 MB | 0.84 s | 1.74 s | 1.7 ms |
Decision (ADR-038): the single-Dapr-key adjacency-list design holds at 10K nodes for the graph structure — traversal is microseconds and the payload is modest. Inlined full-width embeddings do not hold: beyond roughly 1K nodes with production embedding dims, node embeddings belong in the vector store (pgvector/Pinecone/Qdrant/Weaviate adapters) with embedding-free nodes in the Dapr key.
Serve-mode load testing (QPS targets) lives in the grampus-benchmarks
repo per the ADR-036 split.