Skip to main content

​
Overview

Models are now performing long horizon tasks that regularly hit 1m+ tokens. As the conversation prefix grows, cache read costs grow quadratically and eventually outweighs all other token costs. Most information accumulated in agent traces doesn’t need to be kept forever, and clearing it with regular compactions converts the cost of a trace from quadratic to linear. If you perform the compaction when the user is inactive for long enough to miss cache, you can also reduce the severity of cache write costs, as shown below. Tokens billed per turn and total cost, with and without compaction on cache miss

Compacting on cache miss nearly halves the cost of a 50-turn trace. For best results, we recommend keeping the conversation as close to 128k tokens as possible.

For summary-based compactions that take 1-2 minutes, this strategy disrupts the user flow. By using Relace Compact, you get imperceptible compactions that finish within a couple of seconds.

​
Prerequisites

1

Prepare Your Agent Trace

Construct messages exactly as your agent holds them for inference: OpenAI v1/chat/completions, OpenAI v1/responses, or Anthropic v1/messages. The format is detected automatically by the API and returned in the same format.
2

Call the Compact Endpoint

import requests

response = requests.post(
    "https://compact.endpoint.relace.run/v1/code/compact",
    headers={"Authorization": "Bearer [YOUR_API_KEY]"},
    json={
        "messages": messages,
        "target_tokens": 25000,
        "agent_model": "claude-fable-5",
    },
)
result = response.json()

const response = await fetch("https://compact.endpoint.relace.run/v1/code/compact", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${apiKey}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    messages: messages,
    target_tokens: 25000,
    agent_model: "claude-fable-5"
  })
});

const result = await response.json();
The parameter target_tokens sets the token budget for the retained context (defaults to 128k), and the optional agent_model tells us which model generated the trace so we can apply model-specific improvements and count tokens more accurately.
3

Use the Response

{
  "messages": [ ... ],
  "usage": {
    "prompt_tokens": 135407,
    "completion_tokens": 17323,
    "total_tokens": 152730
  }
}
The returned messages is the compressed trace, in the same format you sent. Tool ids come back verbatim, so it can replace your live message list directly.