Caching

Caching saves money and time on requests you send more than once: the same classification, the same extraction, the same prompt over the same document. Cached answers aren't charged.

Gateway requests

Turn caching on in the project's settings (Settings → Gateway in the console) or a key's rules:

{ "cache": { "enabled": true, "ttl_seconds": 3600 } }

Two requests share a cache entry when everything that affects the answer is the same: the model, the messages and the generation parameters. The X-Scalaix-Cache response header says hit, miss, or off.

Override caching for one request with a header:

# never use the cache for this call
curl ... -H "X-Scalaix-Cache: off"

# cache this answer for 10 minutes, even if caching is off for the key
curl ... -H "X-Scalaix-Cache: ttl=600"

Endpoints

Each endpoint has its own cache setting in its definition:

{ "cache": { "enabled": true, "ttl_seconds": 86400, "variants": 1 } }

• Entries are keyed by the endpoint's published version and the inputs, so publishing a new version never serves answers from the old prompt.
• variants (1 to 20) keeps up to that many different answers for the same inputs, for endpoints where some variety is wanted (copywriting, suggestions). With 1, the same inputs always get the same cached answer.
• A caller can skip the cache for one call with "options": {"cache": false}.
• The response's cached field is true when the answer came from cache.

Console test runs are never cached.

When not to cache

Don't cache prompts that depend on the time, on randomness you rely on, or on data that changes between calls with identical text. Keep TTLs short for anything you'd be unhappy to see a day later.