billion-context

Overview

billion-context sits between an agent and its model API and rewrites Anthropic or OpenAI streams using acp-kernel compression. It compresses consumed conversation into layered summaries so a single session can run for days, with the model deciding when and what to compress rather than relying on a hard truncation limit. Compression is described as incremental, reversible, and prefix-cache friendly.

Key Features

• Compresses consumed conversation into layered summaries so one context window can support days-long sessions and billions of tokens.
• Works as a proxy between agents and model APIs, parsing Anthropic or OpenAI request shapes and rewriting streaming responses.
• Injects context-management tools: compress, decompress, search_context, and acp_status.
• Keeps compression incremental, reversible, and prefix-cache friendly, with summaries restorable on demand.
• Offers native plugin, launcher, and base URL prefix usage modes across many coding clients.
• Describes a healthy session as keeping a 95–97% prefix-cache hit rate, with compression itself costing ≤2%.

Use Cases

• Long coding sessions that would otherwise exceed a model's context window.
• Reducing token usage and cost during agent interactions.
• Running a single agent session continuously for days or months while processing billions of tokens.
• Adding context compression to clients such as Claude Code, Codex, OpenCode, pi, Gemini CLI, and others.

Getting Started

• Install with npm install -g billion-context --prefix=~/.local on Linux or macOS, or npm install -g billion-context on Windows.
• Choose a usage mode: native plugin with bili plugin install <client>, launcher with bili <client>, or URL prefix by starting bili and prefixing the client base URL with the local proxy's /bili/ path.
• In native plugin mode, supported clients include pi, omp, opencode, dsh, kimi, hermes, and zcode.

Deployment & Requirements

• Installation uses npm; the global package provides the bili command, with bili-proxy kept as an alias.
• On Linux or macOS, a user-level npm prefix avoids sudo and permission errors; the Windows default npm prefix is user-writable.
• Native plugin mode requires a client with a plugin seam; clients without one use launcher or /bili/ prefix modes.
• Some launcher modes use certificate-based MITM or environment-specific base URL rewrites, depending on the client.
• A manual bili start uses port 8787; client lanes use a self-managed zone starting at 18787 and hop by one on collisions.

Before You Adopt

• License: Other. Review its terms before using, modifying, or distributing the project.
• Native plugins and standalone in-process extensions are mutually exclusive; running both causes double compression.
• Claude Desktop Code tab sessions bypass the proxy and therefore get neither bili compression nor native auto-compaction.
• Goose cannot use certificate-based MITM because rustls trusts no CA file; it uses plain-HTTP base URL redirects instead.