---
title: Retries over HTTP
description: What a cold model returns on SIE Serverless, what a cold start looks like, which responses to retry, and retry loops for curl, Python, TypeScript and agents that do not use the SDK.
canonical_url: https://superlinked.com/docs/cloud/retries
last_updated: 2026-11-18
---

The Python and TypeScript SDKs handle everything on this page for you: they wait for a cold model and retry rate-limited calls by default. Read on if you call SIE Serverless with plain HTTP, an OpenAI client library, or from an agent's tool.

## What a cold model returns

A model with no traffic can scale to zero. The first request after that starts it, and until it is ready you get one of two `503 Service Unavailable` answers. Both carry a `Retry-After` header in seconds.

**`PROVISIONING`**: the model's capacity is starting. On the native endpoints:

```http
HTTP/1.1 503 Service Unavailable
Retry-After: 60
X-SIE-Error-Code: PROVISIONING
Content-Type: application/json

{"error": {"code": "PROVISIONING", "message": "Capacity for model 'BAAI/bge-m3' is starting. Retry after the number of seconds in Retry-After."}}
```

On the OpenAI-compatible endpoints the same refusal is an OpenAI error object with `"type": "server_error"` and `"code": "provisioning"`. The `X-SIE-Error-Code: PROVISIONING` header is there on both. [Cold starts and SIE Relay](/docs/cloud/sie-relay/#retry-while-capacity-starts) lists the `Retry-After` range per endpoint and which requests SIE Relay can answer instead.

**`MODEL_LOADING`**: capacity is up and the model is loading onto it. This one comes with a short `Retry-After`, usually a few seconds:

```json
{"detail": {"code": "MODEL_LOADING", "message": "Local model is loading"}}
```

While capacity starts, the gateway can hold a request for a while before it answers. A cold request can therefore take much longer than a warm one and then either succeed or return the `503`. Give each attempt a generous timeout; the SDKs wait up to 150 seconds for a response.

## What a cold start looks like

A cold start looks like an outage to a client that gives up early. Over plain HTTP, a request to a model that is starting goes like this:

1. The first attempts answer `503` with `X-SIE-Error-Code: PROVISIONING` and a `Retry-After` header. Each answer can take a while to arrive, because the gateway can hold the request first.
2. You wait the number of seconds in `Retry-After` and send the same request again. Several `503` answers in a row are normal.
3. One attempt is held until the model is ready and then answers `200 OK`. That attempt takes much longer than a warm call.

So set each attempt's timeout to at least 150 seconds. A client with a 30 or 60 second timeout gives up on the attempt that would have succeeded, and starts over. And keep retrying for several minutes, not several seconds.

## Which responses to retry

Retry when the status is `429` or `503` **and** the response has a `Retry-After` header. Wait that many seconds, then send the same request again. Keep going until you get another status or your own deadline passes.

| Status | Code | Meaning |
| --- | --- | --- |
| `503` | `PROVISIONING` | The model's capacity is starting |
| `503` | `MODEL_LOADING` | The model is loading onto running capacity |
| `503` | `KEY_STATE_UNAVAILABLE` | The gateway has not confirmed the API key yet, for example seconds after you created it |
| `429` | `RATE_LIMIT` | Over a [rate limit](/docs/cloud/rate-limits/) |

Without an `X-SIE-Fallback-Error` header, these responses mean the request did not run: SIE refused it before any model worked on it, so a retry never runs it twice and is safe for generation too.

A `503` that does carry `X-SIE-Fallback-Error` is different. It comes after a failed [SIE Relay](/docs/cloud/sie-relay/#retry-while-capacity-starts) attempt, and that attempt may have run the request upstream before it failed. Retrying encode, score or extract calls is still fine, since running them again gives the same result. For a `POST` that generates output, resend it only when a duplicate run is acceptable.

Do not retry other `4xx` errors, and do not retry `502 MODEL_LOAD_FAILED`: the model cannot load and the answer will not change.

A cold start can take minutes. The SDKs give up after 900 seconds; a deadline of 10 to 15 minutes is a reasonable default for a batch script, and much shorter is fine for an interactive app that can show "warming up".

## A retry loop without the SDK

#### curl

```bash
url="https://api.superlinked.com/v1/encode/BAAI/bge-m3"
body='{"items": [{"text": "hello"}]}'
deadline=$(( $(date +%s) + 900 ))

while :; do
  code=$(curl -sS -o response.json -D headers.txt -w '%{http_code}' --max-time 150 \
    -X POST "$url" \
    -H "Authorization: Bearer $SIE_API_KEY" \
    -H "Content-Type: application/json" \
    -d "$body")
  wait=$(grep -i '^retry-after:' headers.txt | tr -dc '0-9')
  [[ ( $code == 429 || $code == 503 ) && -n $wait ]] || break
  (( $(date +%s) + wait < deadline )) || break
  echo "HTTP $code, retrying in ${wait}s" >&2
  sleep "$wait"
  # Check again before the next attempt, in case the sleep ran long.
  (( $(date +%s) < deadline )) || break
done
echo "HTTP $code"
cat response.json
```

#### Python

```python
import os
import time

import requests

RETRYABLE = {429, 503}

def post_with_retry(url, payload, deadline_s=900):
    deadline = time.monotonic() + deadline_s
    while True:
        resp = requests.post(
            url,
            json=payload,
            headers={"Authorization": f"Bearer {os.environ['SIE_API_KEY']}"},
            timeout=(10, 150),
        )
        retry_after = resp.headers.get("Retry-After", "")
        if resp.status_code not in RETRYABLE or not retry_after.isdigit():
            break
        wait = int(retry_after)
        if time.monotonic() + wait >= deadline:
            break
        print(f"HTTP {resp.status_code} {resp.headers.get('X-SIE-Error-Code')}, retrying in {wait}s")
        time.sleep(wait)
        # Check again before the next attempt, in case the sleep ran long.
        if time.monotonic() >= deadline:
            break
    resp.raise_for_status()
    return resp.json()

result = post_with_retry(
    "https://api.superlinked.com/v1/encode/BAAI/bge-m3",
    {"items": [{"text": "hello"}]},
)
```

The deadline limits when a new attempt can start, not the total run time. Both loops check it before every attempt, including after a sleep, and send nothing once it has passed. An attempt that starts just before the deadline still runs until it finishes or times out. In curl, `--max-time 150` caps the whole attempt at 150 seconds. In Python, `timeout=(10, 150)` caps the connect at 10 seconds and each wait for data at 150 seconds, so it does not cap the attempt's total time.

OpenAI client libraries already retry `429` and `503` and read `Retry-After`, but only a couple of times by default. Raise their retry count (`max_retries` in the OpenAI Python SDK) or wrap the call in a loop like the one above when a cold start is likely.

## A retry loop for chat completions

These loops call `/v1/chat/completions`. They retry `429` and `503` with `Retry-After` like the loop above, print `X-SIE-Error-Code` on every retry, and stop on any other error. The status and headers decide the retry, so a response body that fails to read does not stop it, and the deadline is checked again after each wait. Two differences from the loop above: they wait 5 seconds when `Retry-After` is not a whole number of seconds, and they stop at a `503` that carries `X-SIE-Fallback-Error`, since that generation may already have run. Pass `resend_after_relay_failure=True` in Python or `true` as the second argument in TypeScript if a duplicate run is acceptable.

The Python version uses only the standard library; its `timeout` caps each wait for data, as in the `requests` loop. The TypeScript version needs `fetch` and `AbortSignal.timeout` (Node 18 or later, Bun or Deno), and its timeout caps the whole attempt. It uses top-level `await`, so save it as an ES module such as `chat.mts` and run it with `bun chat.mts`, `npx tsx chat.mts`, or `node chat.mts` on a Node version that runs TypeScript directly.

#### Python

```python
import json
import os
import time
import urllib.error
import urllib.request

URL = "https://api.superlinked.com/v1/chat/completions"
DEADLINE_S = 900  # stop starting new attempts after 15 minutes
TIMEOUT_S = 150  # per attempt: a request to a starting model can be held this long
DEFAULT_WAIT_S = 5  # if Retry-After is not a whole number of seconds

def chat(payload, resend_after_relay_failure=False):
    body = json.dumps(payload).encode()
    deadline = time.monotonic() + DEADLINE_S
    while True:
        request = urllib.request.Request(
            URL,
            data=body,
            method="POST",
            headers={
                "Authorization": f"Bearer {os.environ['SIE_API_KEY']}",
                "Content-Type": "application/json",
            },
        )
        try:
            with urllib.request.urlopen(request, timeout=TIMEOUT_S) as resp:
                return json.load(resp)
        except urllib.error.HTTPError as err:
            status, headers = err.code, err.headers
            try:
                detail = err.read().decode()
            except Exception:  # the status and headers still decide the retry
                detail = ""

        code = headers.get("X-SIE-Error-Code")
        retry_after = headers.get("Retry-After")
        retryable = status in (429, 503) and retry_after is not None
        # After a failed SIE Relay attempt the request may already have run.
        if headers.get("X-SIE-Fallback-Error") and not resend_after_relay_failure:
            retryable = False
        if not retryable:
            raise RuntimeError(f"HTTP {status} {code}: {detail}")

        digits = retry_after.isascii() and retry_after.isdigit()
        wait = int(retry_after) if digits else DEFAULT_WAIT_S
        if time.monotonic() + wait >= deadline:
            raise RuntimeError(f"Gave up at the deadline: HTTP {status} {code}")
        print(f"HTTP {status} {code}, retrying in {wait}s")
        time.sleep(wait)
        if time.monotonic() >= deadline:  # the sleep can run long
            raise RuntimeError(f"Gave up at the deadline: HTTP {status} {code}")

reply = chat({
    "model": "Qwen/Qwen3.5-4B",
    "messages": [{"role": "user", "content": "Say hello in five words."}],
    "max_tokens": 64,
})
print(reply["choices"][0]["message"]["content"])
```

#### TypeScript

```typescript
const URL = "https://api.superlinked.com/v1/chat/completions";
const DEADLINE_MS = 15 * 60_000; // stop starting new attempts after 15 minutes
const TIMEOUT_MS = 150_000; // per attempt: a request to a starting model can be held this long
const DEFAULT_WAIT_S = 5; // if Retry-After is not a whole number of seconds

async function chat(payload: object, resendAfterRelayFailure = false): Promise<any> {
  const body = JSON.stringify(payload);
  const deadline = performance.now() + DEADLINE_MS;
  while (true) {
    const resp = await fetch(URL, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.SIE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body,
      signal: AbortSignal.timeout(TIMEOUT_MS),
    });
    if (resp.ok) return resp.json();

    const code = resp.headers.get("X-SIE-Error-Code");
    const retryAfter = resp.headers.get("Retry-After"); // null when absent
    let retryable = (resp.status === 429 || resp.status === 503) && retryAfter !== null;
    // After a failed SIE Relay attempt the request may already have run.
    if (resp.headers.get("X-SIE-Fallback-Error") && !resendAfterRelayFailure) {
      retryable = false;
    }
    // The status and headers still decide the retry if the body fails to read.
    const detail = await resp.text().catch(() => "");
    if (!retryable) throw new Error(`HTTP ${resp.status} ${code}: ${detail}`);

    const wait = /^\d+$/.test(retryAfter ?? "") ? Number(retryAfter) : DEFAULT_WAIT_S;
    if (performance.now() + wait * 1000 >= deadline) {
      throw new Error(`Gave up at the deadline: HTTP ${resp.status} ${code}`);
    }
    console.log(`HTTP ${resp.status} ${code}, retrying in ${wait}s`);
    await new Promise((resolve) => setTimeout(resolve, wait * 1000));
    if (performance.now() >= deadline) { // the sleep can run long
      throw new Error(`Gave up at the deadline: HTTP ${resp.status} ${code}`);
    }
  }
}

const reply = await chat({
  model: "Qwen/Qwen3.5-4B",
  messages: [{ role: "user", content: "Say hello in five words." }],
  max_tokens: 64,
});
console.log(reply.choices[0].message.content);
```

## For agent tools

Agents that call SIE Serverless through a generic HTTP tool often treat the first `503` as a failure. Paste this into the tool's description or the system prompt:

> If SIE returns HTTP 503 or 429 with a `Retry-After` header, the model is starting or you sent too many requests. Unless the response also has an `X-SIE-Fallback-Error` header, the request did not run. Wait the number of seconds in `Retry-After`, then send the identical request again. If `X-SIE-Fallback-Error` is present and the request generates text, the first attempt may have run, so resend it only when a second run is acceptable. Repeat until you get a different status, for up to 15 minutes. Do not change the model or the request, and do not report a failure while these retries are still going. Send several items in one call instead of one call per item.

If your agent framework can run Python, give it the SDK instead; it does all of this by default.

## Using a coding agent?

When a coding agent writes the client code for you, paste these lines into its instructions (`AGENTS.md`, `CLAUDE.md` or your tool's equivalent) so the code it writes waits out a cold start:

```text
SIE Serverless returns HTTP 503 with a Retry-After header (in seconds) while a model starts.
Wait that many seconds and send the same request again. Keep retrying for up to 15 minutes;
a cold start can take minutes, so several 503s in a row are expected.
Set the HTTP client timeout to at least 150 seconds, because a request to a starting model
can be held that long before it answers.
HTTP 429 means too many requests: slow down, wait Retry-After seconds, then retry.
Retry a 503 or 429 only when it has a Retry-After header; without one, treat it as an error.
If Retry-After is present but is not a whole number of seconds, wait 5 seconds.
Do not retry other 4xx errors. If a 503 also has an X-SIE-Fallback-Error header, do not
resend a generation request unless a duplicate run is acceptable.
Retry code and details: https://superlinked.com/docs/cloud/retries/
```
