Pre-warming

Novo's hosted runtime scales to zero. When no run has touched a thread for about a minute, the next message has to cold-start the infrastructure before the model can respond, which shows up as a few seconds of silence before the first token. Pre-warming lets you pay that cost ahead of the user, so the response feels instant.

Why the pause happens

A message travels through two independently-scaled serverless functions:

The API route that accepts the request and starts the run.
The durable execution worker that actually runs the agent loop (so a run survives restarts, retries, and long tool calls).

Under steady traffic both stay warm and the hop between them is milliseconds. But with sparse or bursty traffic (the early life of most apps, or any thread a user returns to after a break) each can be cold, and a cold start adds several seconds before the provider is even called. This is the unavoidable flip side of scale-to-zero: you don't pay for idle capacity, so the first request after idle pays to bring capacity up. The first turn's data-novo-step-terminal perf fields separate that pickup/prepare time from model time in your own traces.

The fix: warm on intent

You almost always know a user is about to send a message a few seconds before they actually do: they focused the composer, opened the thread, or started typing. Fire a pre-warm on that signal:

// In your composer, on focus / open / first keystroke:
void novo.threads.prewarm(thread.id);

prewarm brings both functions up so they're hot by the time the real message lands. It's a free no-op: no run is created, nothing is billed, so you can call it liberally; rapid repeats are coalesced server-side, so over-firing is harmless.

The single best moment to warm is a brand-new conversation: the user just arrived, no prior run is keeping anything hot, and you don't have a thread id yet. Call prewarm with no argument for that; warming targets shared infrastructure, not a specific thread:

// Before a thread exists, e.g. when a fresh composer opens:
void novo.threads.prewarm();

The window: aim for a 10–30 second lead

Pre-warming has a floor and a ceiling:

Floor (~10s). The warm itself spends a few seconds bringing the path up, and prewarm returns as soon as the warm is in flight rather than waiting for it. So measure the lead from when you fire prewarm, and treat ~10s as the point where the benefit is reliably there; a message landing sooner gets only partial benefit.
Ceiling (~60s). Warmth lapses after roughly a minute of idle. A warm fired more than ~30s before the message risks going cold again.

So the sweet spot is 10–30 seconds before the message. Because it's free, the simplest robust strategy is to fire on the earliest intent signal and again on a throttled interval while the user keeps composing:

// `prewarm` is your bound call; `throttle` is any throttle helper (lodash, or a
// few-line timestamp guard). Pre-warm is free, but there's no point firing it on
// every keystroke; throttle to ~once per 20s.
const prewarm = () => void novo.threads.prewarm(thread.id);

onFocus={prewarm}
onChange={throttle(prewarm, 20_000)}

After any message runs, the thread stays warm for ~60s on its own, so follow-up turns in an active conversation need no pre-warm. Pre-warming matters for the first message after a quiet period.

When it helps, and when it doesn't

Pre-warming shines in interactive UIs, where a human's natural read-and-type time comfortably fills the 10–30s lead.

It does little for programmatic or batch callers that have no human pause before a request: there's no organic lead to fire it in, and pinging exactly ~10s ahead of a server-driven call is rarely worth the orchestration. Those workloads are better served by keeping a steady request rate (which keeps the runtime warm on its own) or simply absorbing the occasional cold start.

This is also, by design, a bridge for low-traffic stages. As an app gains sustained traffic, the runtime stays warm without help and cold starts mostly disappear.

Raw HTTP

Not using the SDK? Issue a GET to the messages route with ?warm=1 and your API key:

curl "https://api.novoagents.ai/v1/threads/$THREAD_ID/messages?warm=1" \
  -H "Authorization: Bearer $NOVO_AGENTS_API_KEY"
# → 204 No Content