Continuous optimization
The counterfactual is the point: lenops keeps a baseline of what each request would have cost under its original model and context, then charts it against what was actually spent. The gap between the two lines is the product.
Saved
$487.00
24.8% below baseline
Actual spend
$1473.00
14d cumulative
Baseline
$1960.00
no intervention
Run rate
−$34.79/day
net change vs baseline
Projected annual
$12694.00
at current rule set
Counterfactual
Optimized spend vs spend without intervention
Two cumulative lines. The first step change is routing; the second is the optimization rule set.
actual baseline (no intervention) cumulative saving
Attribution
Where the saving came from
| Source | Kind | Saved | Share |
|---|---|---|---|
| routing · routine-and-non-dev → local | routing | $218.40 | 44.8% |
| optimize · context_compaction | optimization | $104.22 | 21.4% |
| optimize · prompt_cache | optimization | $81.16 | 16.7% |
| optimize · model_routing (downshift) | optimization | $52.90 | 10.9% |
| routing · availability fallbacks | routing | $30.32 | 6.2% |
Rules
Optimization rule set
4 active · 3 enforcing, 1 in shadow. Shadow rules measure the saving without changing the response.
cache-repeat-prompts
prompt_cache · when prompt repeated ≥2x
compact-long-context
context_compaction · when prompt tokens > 8000
downshift-on-pressure
model_routing · when team budget used > 85%
batch-ci-docs
request_batching · when queue depth > 5
Safety
Rails that no rule may cross
Checked before any strategy is applied. A rule that would violate a rail is skipped and the skip is logged.
- ✓never degrade priority=p0
- ✓never compact below the declared minimum context
- ✓never apply a lossy strategy to environment=prod unless the rule opts in explicitly
Catalog
Available strategies
Ten strategies the harness can apply. Two of them optimize latency rather than cost, and are labelled as such.
| Strategy | What it does | Typical saving | In use |
|---|---|---|---|
| prompt_cache | reuse a cached prefix | 60–90% of cached tokens | active |
| context_compaction | summarize history beyond a threshold | 30–60% of prompt tokens | active |
| model_routing | serve from a cheaper equivalent tier | varies by tier gap | active |
| request_batching | coalesce queued requests | 10–25% overhead | active |
| response_cache | return a stored answer for an exact hit | 100% of the call | — |
| max_tokens_cap | cap output tokens per team or workload | 5–20% of output tokens | — |
| retrieval_pruning | lower RAG top-k and dedupe chunks | 10–30% of prompt tokens | — |
| tool_iteration_cap | limit agentic tool iterations per task | 10–35% per task | — |
| streaming | stream tokens to cut time-to-first-token | latency only | — |
| speculative_decoding | draft model proposes, larger model verifies | latency only | — |