Quickstart

Overview

Models are now performing long horizon tasks that regularly hit 1m+ tokens. As the conversation prefix grows, cache read costs grow quadratically and eventually outweighs all other token costs. Most information accumulated in agent traces doesn’t need to be kept forever, and clearing it with regular compactions converts the cost of a trace from quadratic to linear. If you perform the compaction when the user is inactive for long enough to miss cache, you can also reduce the severity of cache write costs, as shown below.

Tokens billed per turn and total cost, with and without compaction on cache miss

Compacting on cache miss nearly halves the cost of a 50-turn trace. For best results, we recommend keeping the conversation as close to 96k tokens as possible.

For summary-based compactions that take 1-2 minutes, this strategy disrupts the user flow. By using Relace Compact, you get imperceptible compactions that finish within a couple of seconds.

Prerequisites

Sign up for a Relace account.
Create an API key.

Prepare Your Agent Trace

Construct messages exactly as your agent holds them for inference: OpenAI v1/chat/completions, OpenAI v1/responses, or Anthropic v1/messages. The format is detected automatically by the API and returned in the same format.

Call the Compact Endpoint

import requests

response = requests.post(
    "https://models.relace.ai/v1/code/compact",
    headers={"Authorization": "Bearer [YOUR_API_KEY]"},
    json={
        "messages": messages,
        "target_tokens": 96000,
        "agent_model": "claude-fable-5",
    },
)
result = response.json()

const response = await fetch("https://models.relace.ai/v1/code/compact", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${apiKey}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    messages: messages,
    target_tokens: 96000,
    agent_model: "claude-fable-5"
  })
});

const result = await response.json();

The parameter target_tokens sets the approximate token budget for the retained context (defaults to 96k), counted in your agent model’s tokens. agent_model tells us which model generated the trace, so we can apply model-specific improvements and count the budget in the right tokenizer.

Use the Response

{
  "messages": [ ... ],
  "usage": {
    "prompt_tokens": 135407,
    "completion_tokens": 17323,
    "total_tokens": 152730
  }
}

The returned messages is the compressed trace, in the same format you sent. Tool ids come back verbatim, so it can replace your live message list directly.

Usage Tips

It makes the most sense to perform compaction in the following cases:

On cache miss Provider prompt caches have a finite TTL after which you are charged the full cache write cost (5 min or 1 hr for Anthropic). Compacting when a user comes back to a stale session is a strict win — you reduce your cache write costs while also reducing the cache read costs for future messages. Wait until it has been more than 10 minutes since the user’s last message, comfortably past the cache TTL, so a fresh compaction doesn’t immediately re-fire.
Only when it pays off Let the context grow at least 64k tokens above your compact window before compacting, so there’s a meaningful amount to reclaim. With the recommended 96k window, that means compacting once you cross ~160k tokens.
Near the context limit This is already necessary for all agent harnesses. If an active session exceeds the model’s context window (usually 1m tokens for Anthropic), you must implement some compaction to continue the session.

Additionally, you should always compact at a clean turn boundary (no tool call open), before adding the newly-arrived user message.

Getting Started

Model Guides

Repos

Agents

Overview

Prerequisites

Usage Tips

​Overview

​Prerequisites

​Usage Tips

Overview

Prerequisites

Usage Tips