Why do AI coding agents remember everything but forget nothing?

Concept: Context Resource Management
Status: Open Source Tool (PyPI)

---

The Question

As conversations with AI coding agents (Claude Engineer, Aider, Claude Code) progress, they accumulate files in their context window. Since models retrieve info from all loaded data, stale files pollute the context window, causing latency, hallucination spikes, compilation errors, and cost growth.

How do we build a system that dynamically monitors context pressure, scoring files by age and relevance, and automatically compressing or evicting them?

---

Constraints

• Strict Token Boundaries: Exceeding model limits triggers hard API errors.
• Semantic Preservation: Evicting a helper class entirely breaks compilation if other files call it. We must compress files (leaving signature templates) before evicting them.
• Character Offset Boundaries: Parsing and truncating source files containing multi-byte UTF-8 characters easily causes alignment crashes if slice indexes split character boundaries.

---

Tradeoffs: Full Eviction vs. AST Compression

When context pressure exceeds the safe threshold, the engine must free space. We chose between:

• Greedy Eviction (Deletion): Dropping the oldest files entirely from context. This is simple, but breaks references if the model needs helper signatures.
• AST Compression (Symbol Stripping): Using tree-sitter S-expression queries to parse code and prune internal function bodies, leaving only class definitions, function signatures, and comments.

We chose a hybrid model. The evictor uses Turn-decay (a mathematical sigmoid decay curve representing active file age) to score staleness. When pressure rises, it first runs AST compression on compressible code to shrink its size by 60%, and only falls back to Greedy Eviction for non-compressible files or extreme pressure overflows.

---

Interactive Simulation

Move the slider to increase token pressure and watch the eviction pipeline dynamically compress Python/TypeScript source files or evict non-compressible config nodes:

---

Architecture & Implementation Details

Byte-Space String Slicing in compressor.py

Tree-sitter returns syntax tree offsets in raw bytes, not character indexes. In multi-byte character UTF-8 environments, slicing Python strings directly with byte offsets causes index mismatch crashes.

To solve this, we perform all index calculations and string slicing in byte space:

# Slicing safely in byte space
def slice_source_bytes(source_str: str, intervals: list[tuple[int, int]]) -> str:
    # Encode to bytes first
    source_bytes = source_str.encode("utf-8")
    result_bytes = bytearray()
    
    for start, end in intervals:
        result_bytes.extend(source_bytes[start:end])
        
    # Decode back to character string at the boundary
    return result_bytes.decode("utf-8")

---

Lessons Learned & Retrospectives (The Integration Gaps)

Our code audit revealed two integration bugs that bypass the flagship AST compression features:

1. The Dead Compression Tool Bug: In mcp_server.py, the optimize_context server tool runs evaluate but omits passing the file_contents parameter. Because file_contents is left as None, the evaluator skips compression checks and defaults directly to eviction recommendations.
2. Hardcoded Extensions: In evictor.py, _try_compress is hardcoded to only run tree-sitter on files ending in .py. As a result, the AST pruner queries written for JavaScript, TypeScript, and Go are completely bypassed in the active eviction evaluation loops.
3. Takeaway: Algorithmic core logic is only as valuable as the integration testing that wraps it. We are building integration tests in Version 0.2 to mock the server context directly.