Chat

Multi-turn chat

There is no session, thread or conversation id, and no store flag. Every request stands alone: you send the whole messages array, and what comes back is a completion rather than a handle to resume from. Your client owns the conversation — keep the array, append each turn to it, and send it again.

import { generateText, type ModelMessage } from 'ai';

const model = faucet('anthropic/claude-opus-5');
const messages: ModelMessage[] = [
  { role: 'user', content: 'Explain a leaky bucket rate limiter in two sentences.' },
];

const first = await generateText({ model, messages });
messages.push(...first.responseMessages); // the assistant's turn

messages.push({ role: 'user', content: 'Now contrast it with a fixed window.' });
const second = await generateText({ model, messages });

Against the wire format directly it is the same move in OpenAI's own shape: push the choices[0].message you were given, then the next user message after it.

This is a design decision rather than a missing feature. A conversation you can resume by id is a conversation somebody is storing, and Faucet records what a request cost, never what it said — there is no payload table here that could hold your history. Keeping the state on your side is what lets that stay true.

Two things to plan for as a thread grows:

• You pay for the history on every turn unless it is cached. The next section is how to make it cached.
• The context window is the ceiling, and it is per request, not per conversation. Overrun it and the provider's own 400 reaches you with its message intact. Summarize or drop old turns before that point; the window for each model is in the Models reference.

Caching a long prompt

A growing conversation, a long system prompt, a document you ask ten questions about — all the same shape: a large stable prefix, re-sent every turn. Mark where that prefix ends and the provider stores it, so later requests read it back instead of paying for it again.

The marker is cache_control, on a message or on a single content part:

{
  "model": "anthropic/claude-opus-5",
  "messages": [
    {
      "role": "system",
      "content": "<long, unchanging instructions>",
      "cache_control": { "type": "ephemeral" }   // cache everything to here
    },
    { "role": "user", "content": "the question, which changes every turn" }
  ]
}

Put it on the last message that will not change. Everything before and including it is cached; everything after is priced normally. In a chat loop that usually means moving the marker forward as the thread grows — onto the last assistant turn before the new question.

ttl is optional and takes "5m" or "1h". The longer window costs more to write and is worth it for a prefix that outlives a single burst of activity.

Reading a cached prefix is cheaper than sending it fresh; writing it costs more than sending it once (Anthropic charges 1.25×). So caching pays off from the second request onward, and a prefix used once is worse than no caching at all. Both numbers come back in the response, and both are metered and billed as their own line rather than as ordinary input:

"usage": {
  "prompt_tokens": 4210,
  "prompt_tokens_details": {
    "cached_tokens": 4096,           // read back — the cheap ones
    "cache_creation_tokens": 0       // written this request — the 1.25× ones
  }
}

cache_creation_tokens is a Faucet addition, since OpenAI's format has no field for it. Clients that do not know it ignore it.

Three things to know before relying on it:

• Anthropic honours the marker; Google ignores it. Google caches automatically instead, so there is nothing to mark — the field is accepted and does nothing rather than failing your request, which keeps a model swap a one-string change. On the providers Faucet passes through untouched (OpenAI, Fireworks), whatever that provider does with the field is what happens; OpenAI caches automatically too.
• Four breakpoints, maximum. Anthropic allows four; markers past the fourth are ignored. One at the end of your stable prefix is the usual answer.
• The prefix must match exactly, byte for byte, from the start of the request. One edited character in your system prompt invalidates everything after it, which is why the marker belongs after the parts that never change — not after a timestamp you inject each turn.

Tools, JSON and image inputs

Tool calls (including parallel ones), tool results, JSON mode and JSON schema, image inputs and reasoning text all survive a round trip on every provider. Reasoning is emitted as reasoning_content.

A few OpenAI parameters only work on providers Faucet can pass through byte-for-byte, and not on the two whose wire format has to be translated (Anthropic and Google):

provider_options is the escape hatch for provider-native features the OpenAI format cannot express. It is keyed by provider slug, so options meant for one upstream can never leak into a request to another:

{
  "model": "anthropic/claude-opus-5",
  "messages": [{ "role": "user", "content": "…" }],
  "provider_options": {
    "anthropic": { "thinking": { "type": "adaptive" } }
  }
}