---
title: Usage and token counting
description: Read usage from a standalone SIE server for encode, score, extract, and generate, then understand the differences when deploying a cluster.
canonical_url: https://superlinked.com/docs/deployment/usage
last_updated: 2026-09-17
---

SIE measures the work an inference request performs. Your self-hosted application can use those measurements to apply its own prices and produce invoices. You own the customer ledger, payment integration, and billing policy; running the open-source engine does not require Superlinked Cloud or Metronome.

**A token count is a quantity. It becomes a charge only when your billing service applies a price.** Keep the measured quantities so a price change does not erase the evidence behind an earlier invoice.

Start with the standalone server below. If you deploy a cluster, also read [Cluster and gateway differences](#cluster-and-gateway-differences): the gateway does not expose every field that a standalone server does. For latency, capacity, and dashboards, see [Monitoring and observability](/docs/deployment/monitoring/).

## Standalone server usage

A standalone `sie-server` accepts the request, prepares the model's input, and builds the HTTP response from the inference result. The sections below describe each native operation's usage and SDK access separately.

```text
Your application sends an encode, score, extract, or generate request
  → sie-server prepares inputs and runs the selected model
  → The HTTP response includes the available usage fields
  → Your application records those quantities and applies its prices
```

The [public model catalog](https://github.com/superlinked/sie/tree/main/packages/sie_server/models) covers text, image, video, audio, and document inputs. Choose the relevant operation and deployed profile; a model name alone does not establish measurement coverage. The `sie-fake` catalog fixture produces synthetic results and is not evidence for a real model.

The numerical examples on this page are illustrative, not benchmark results or claims about how a particular sentence tokenizes. **Missing usage means no measurement is available to the caller.** Never replace an absent field with zero or label a character estimate as a measured token count.

### Encode

Source: [packages/sie_server/src/sie_server/api/encode.py](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/api/encode.py)

Native encode can return `usage.input_tokens`, summed across the input items. It adds `usage.images` when complete image counts are also available. If token counts are unavailable or incomplete, this usage block is omitted; a measured token count of zero is preserved.

Text counts come from the preprocessor or adapter, with a supported adapter-tokenizer fallback. They follow the applicable truncation limit, include model-specific special tokens, and exclude batch padding. Instructions and tokenizer choice can change the count for the same text.

| Model family | Measurement | Counting rule |
| --- | --- | --- |
| Dense, sparse, and hybrid text embeddings, such as E5, SPLADE, and BGE-M3 | `input_tokens` when available | Count processed input tokens with applicable prefixes, special tokens, and length limits. Vector dimensions, sparse nonzero entries, and the number of output formats are not additional token counts |
| Text multi-vector embeddings, such as ColBERT | `input_tokens` when available | Use the adapter's token count. Query and document preprocessing can differ; the number of returned vectors is not a universal substitute |
| Image/text and visual-document embeddings, such as CLIP, SigLIP, ColPali, ColQwen, ColSmol, ColEmbed, and V-SPLADE | `images`, plus `input_tokens` where supplied | Keep processed images and text tokens separate. A page supplied as an image follows the image encoder's contract; it does not automatically become a Docling page unit |
| Video-aware embeddings, such as Qwen3-VL-Embedding | `images`, plus available `input_tokens` | Sampled frames count as processed images. Count the frames the adapter used, rather than one unit for the uploaded video or every frame in its source file |

**Example: two text inputs.** If the processed inputs measure 3 and 11 tokens, the native HTTP response contains this usage excerpt:

```json
{
  "usage": {
    "input_tokens": 14
  }
}
```

Requesting both dense and sparse vectors does not change that total to 28. Output dimensions and vector formats are not additional input tokens. For a video-aware encoder, eight sampled frames can produce an image count of eight; the source file count is one, and only available fields reach the response.

**SDK access:** encode convenience calls return item results, not the complete native response envelope. Do not assume `client.encode(...)` provides a top-level usage summary. Your billing integration needs an export that retains the available native usage fields. See the [Python SDK reference](/docs/reference/sdk/) and [TypeScript SDK reference](/docs/reference/typescript-sdk/) for item result shapes.

**OpenAI-compatible embeddings:** standalone `/v1/embeddings` can substitute a character estimate for missing usage, without an annotation identifying that fallback. Use native encode's measured usage for token-based accounting on this topology. A scheduling estimate such as `len(text) / 4` is not a model-tokenizer count.

### Score

Source: [packages/sie_server/src/sie_server/api/score.py](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/api/score.py)

Native score can return `usage.input_tokens` across all evaluated query/document pairs, plus `usage.images` when supplied. It reads these counts from the adapter output. Missing token counts omit the usage block; a measured zero is preserved.

| Model family | Measurement | Counting rule |
| --- | --- | --- |
| Text cross-encoder rerankers, including BGE, Jina, and Qwen | `usage.input_tokens` when supplied | Each query/document pair is tokenized with model-specific separators and instruction handling. The query contributes to each evaluated pair. If the adapter supplies no token count, the standalone response omits usage |
| Late-interaction rerankers, such as ColBERT and BGE-M3 | `usage.input_tokens` when available | Use adapter-specific query and document tokenization. Their limits can differ. Client-side MaxSim over previously encoded vectors does not send another inference request to SIE |
| Vision-language reranking, such as Qwen3-VL-Reranker | `usage.input_tokens`, optional `usage.images` | Count the processor's input IDs after document truncation and before padding. Its count includes the multimodal prompt representation. Count query and document images consumed by each pair; a reused query image contributes again for each candidate |
| Stablebridge reranking | No token usage | Its current score adapter supplies no token counts, and its inherited counting hook returns `None`. A relevance score does not establish token usage |

**Example: one query and two documents.** If the complete pairs measure 18 and 26 tokens, request usage is `input_tokens = 44`. Both counts already include the query. Do not add its tokens a third time or charge the same request-level total for every returned score.

For vision-language reranking, one query image compared with two document images consumes four images across the two pairs: the query image is used twice. Processor token counts remain a separate quantity.

#### Score SDK example

The `score` SDK result preserves an optional usage summary. This example uses `BAAI/bge-m3` with its default profile, whose adapter supplies query-plus-document token counts directly through the [BGE-M3 scoring implementation](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/bge_m3_score_mixin.py). Enable that model in your deployment before running it.

The example rejects unavailable usage and accepts a measured zero. It reads usage once for the whole request. Other rerankers can differ: the default `BAAI/bge-reranker-v2-m3` adapter supplies no token counts to the standalone score response.

#### Python

```python
from sie_sdk import SIEClient

with SIEClient("http://localhost:8080") as client:
    result = client.score(
        "BAAI/bge-m3",
        {"text": "How do I count tokens?"},
        [
            {"text": "Use the model tokenizer's measured counts."},
            {"text": "An embedding is a vector representation."},
        ],
    )

usage = result.get("usage")
if usage is None:
    raise RuntimeError("Usage was unavailable; reconcile this attempt")

print("Input tokens across all pairs:", usage["input_tokens"])
```

#### TypeScript

```typescript
import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");
try {
  const result = await client.score(
    "BAAI/bge-m3",
    { text: "How do I count tokens?" },
    [
      { text: "Use the model tokenizer's measured counts." },
      { text: "An embedding is a vector representation." },
    ],
  );

  if (result.usage === undefined) {
    throw new Error("Usage was unavailable; reconcile this attempt");
  }

  console.log("Input tokens across all pairs:", result.usage.inputTokens);
} finally {
  await client.close();
}
```

### Extract

Source: [packages/sie_server/src/sie_server/api/extract.py](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/api/extract.py)

Native standalone extract can return `usage.input_tokens` when the adapter supplies valid per-item token counts. It does not put image counts, page counts, or audio duration into this public usage block. A model can return useful entities, OCR text, or a transcript without exposing a usage quantity to the caller.

| Model family | Standalone response | Counting rule or current gap |
| --- | --- | --- |
| Classic GLiNER entity extraction | `usage.input_tokens` | Retained document subwords after the processor's truncation. Label-prompt words and padding are excluded; tokenizer special tokens remain counted |
| GLiNER2 entities, relations, structured extraction, and classification, including GLiGuard; GLiClass classification | `usage.input_tokens` when available | Count document-only tokenizer IDs with normal special tokens and the configured length cap. The label/schema prompt is excluded from this meter; the number of returned labels or entities is not the usage quantity |
| GLiNER-bi entities, GLiREL relations, NLI classification, and Stablebridge extraction/pruning/highlighting | No token usage | Useful results can arrive without a token measurement. GLiNER-bi does not inherit classic GLiNER's measured-count behavior. Pruned/kept spans and classification scores do not establish processed-token usage |
| Captioning, detection, visual QA, and image-based OCR, including Florence-2, OWLv2, Grounding DINO, Donut, GLM-OCR, PaddleOCR-VL, and MinerU | No image usage field | Use one image per item; see the [image-count limitation](#image-count-limitation). Output boxes, captions, and OCR characters are results, not token measurements. An adapter that internally generates text does not necessarily expose generation usage through `extract` |
| LightOnOCR, including its SGLang profile with `meter_pages: true` | No page usage field | One processed image represents one page per extraction item. This explicit page meter takes precedence over a generic image count |
| Docling document parsing and OCR | No page usage field | Uses the processed-page list, with document metadata as a fallback for older result shapes. Docling returns `0` if page metadata cannot be read, so zero alone cannot distinguish no processed pages from missing evidence |
| Whisper audio transcription through shared audio preparation | No audio-duration usage field | Source sample count divided by source sample rate, rounded up to whole milliseconds. This measures accepted audio duration, not inference runtime or transcript tokens |

**Example: GLiNER2 classification.** A document measured at 120 tokens and classified against five labels produces this native HTTP usage excerpt:

```json
{
  "usage": {
    "input_tokens": 120
  }
}
```

The document-only meter does not multiply 120 by the number of labels. By comparison, if GLiREL returns two relations without token counts, usage remains unavailable. Two relations establish neither two processed tokens nor zero processed tokens.

**SDK access:** extract convenience calls return item results, not the complete native response envelope. They are not a universal export of `usage.input_tokens`. Preserve the native usage through an explicit measurement export if your billing integration needs it. Page, image, and audio billing also require such an export; their [cluster worker units](#additional-worker-units) do not automatically become public HTTP fields.

### Generate

Native generation reports `usage.prompt_tokens`, `usage.completion_tokens`, and `usage.total_tokens`. These quantities come from the generation backend. They are distinct from the encode, score, and extract field named `input_tokens`.

| Model family | Measurement | Counting rule |
| --- | --- | --- |
| Text generation, chat, structured JSON, and generative guardrails | `prompt_tokens`, `completion_tokens` | Use generation-backend counts. Chat templates, instructions, and tokenization affect prompt usage; JSON fields and classification labels do not replace output-token counts |
| Vision-language generation, such as image-capable Qwen and Gemma profiles | Backend-reported `prompt_tokens`, `completion_tokens` | Use the engine's multimodal usage report. There is no universal image-to-token multiplier, and the generation usage envelope does not provide a separate `images` total |
| Translation with MADLAD/CTranslate2 | `prompt_tokens`, `completion_tokens` | Input comes from the translation tokenizer; output is counted from generated token sequences. The target-language prefix can affect the input count |

#### Generation example and streaming

Source: [packages/sie_server/src/sie_server/adapters/sglang/generation.py](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/sglang/generation.py)

The SGLang adapter reads `prompt_tokens` and `completion_tokens` from the engine's terminal usage metadata. Use those counts rather than re-tokenizing the displayed answer: formatting, chat templates, and model-specific token handling can differ from what is visible in the response text.

Native generation reports:

```json
{
  "usage": {
    "prompt_tokens": 2000,
    "completion_tokens": 300,
    "total_tokens": 2300
  }
}
```

This is an illustrative response excerpt. `total_tokens` is the sum of the other two fields. If your prices distinguish input and output, price those fields separately; adding the total again would double-count the request. `max_new_tokens` is an output limit, not the amount generated.

For streaming, native `generate` puts usage on the terminal chunk. OpenAI-compatible chat/completions streams require `stream_options.include_usage: true` for their usage chunk. A stream chunk can contain part of a token or several tokens, so counting chunks does not measure tokens. An interrupted stream may never deliver final usage to the client; record an incomplete attempt rather than inferring a zero charge.

OpenAI-compatible chat/completions use the same prompt/completion/total names. The Responses API uses `input_tokens`, `output_tokens`, and `total_tokens` for its generation summary.

A failed standalone stream can contain zero defaults when final counts never arrived. Record its failure status and reconcile the attempt before deciding a charge.

#### Generate SDK example

Generation SDK parsing can lose the distinction between missing and zero. Python omits `usage` when the entire block is missing, but converts missing token fields inside a supplied block to zero. TypeScript returns zero counts even when the entire block is missing. For auditable billing, your measurement export must preserve whether counts were actually supplied; a typed SDK result alone cannot prove this. See the [Python parser](https://github.com/superlinked/sie/blob/main/packages/sie_sdk/src/sie_sdk/client/sync.py) and [TypeScript parser](https://github.com/superlinked/sie/blob/main/packages/sie_ts_sdk/src/internal/parsing.ts).

Source: [packages/sie_sdk/src/sie_sdk/client/sync.py](https://github.com/superlinked/sie/blob/main/packages/sie_sdk/src/sie_sdk/client/sync.py)

Use a generation model enabled in your deployment whose usage reporting you have verified. These examples read the SDK fields; the completion check does not establish that the backend supplied measured counts.

#### Python

```python
from sie_sdk import SIEClient

with SIEClient("http://localhost:8080") as client:
    result = client.generate(
        "Qwen/Qwen3.5-4B",
        "Explain vector search in one sentence.",
        max_new_tokens=64,
    )

if result.get("finish_reason") not in {"stop", "length"}:
    raise RuntimeError("Record this attempt for failure reconciliation")

usage = result.get("usage")
if usage is None:
    raise RuntimeError("Usage was unavailable; reconcile this attempt")

print("Input tokens:", usage["prompt_tokens"])
print("Output tokens:", usage["completion_tokens"])
```

#### TypeScript

```typescript
import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");
try {
  const result = await client.generate(
    "Qwen/Qwen3.5-4B",
    "Explain vector search in one sentence.",
    { maxNewTokens: 64 },
  );

  if (!["stop", "length"].includes(result.finishReason)) {
    throw new Error("Record this attempt for failure reconciliation");
  }

  console.log("Input tokens:", result.usage.promptTokens);
  console.log("Output tokens:", result.usage.completionTokens);
} finally {
  await client.close();
}
```

## Cluster and gateway differences

Source: [packages/sie_gateway/src/handlers/proxy.rs](https://github.com/superlinked/sie/blob/main/packages/sie_gateway/src/handlers/proxy.rs)

In a queue-mode cluster, the [request pipeline](/docs/engine/) adds a gateway and worker result transport. The adapter's measurement rules still apply, but the gateway builds the public response from worker results. Do not assume the standalone response contract carries through unchanged.

```text
Your application → gateway → queued work → worker
  → worker result and per-work-item units → sidecar → gateway
  → public response with the usage fields supported for that operation
```

### Public response differences

| Operation | Cluster behavior |
| --- | --- |
| `encode` | The gateway sums `input_tokens` only when every included successful result supplies a token count. It includes `images` only when every included result also supplies that field. A missing required count prevents an incomplete aggregate from being reported |
| `score` | The same aggregate rules apply. Before reporting results, the worker can recover missing pair token counts with the adapter's tokenizer. This is why the default `BAAI/bge-reranker-v2-m3` can supply token usage on the queue path even though its standalone response omits usage |
| `extract` | The current gateway response builder does not add aggregate `usage`, even when the worker measured tokens, pages, images, or audio duration. A standalone extract token-usage integration therefore needs a separate export when moved to this path |
| `generate` | The gateway uses the generation result's prompt/completion/total usage. Native and compatible streaming still require final usage to arrive; SDK parsing can still lose missing-versus-zero information |

**Zero in queue encode:** the worker emits a zero token count only when that item also reports a positive image count. It drops a zero without that image evidence, which can leave the gateway without enough counts to produce aggregate usage. This differs from standalone encode's preservation of measured zeros.

**Gateway `/v1/embeddings`:** the compatibility response adds `sie_token_source` alongside `prompt_tokens` and `total_tokens`. `"worker"` identifies a measured count; `"character_estimate"` identifies a fallback estimate. Preserve that source in your records. The standalone compatibility endpoint does not supply this annotation.

### Additional worker units

Source: [packages/sie_server/src/sie_server/queue_executor.py](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/queue_executor.py)

Workers can report `pairs`, `pages`, `images`, and `audio_ms` internally. These are not all exposed uniformly in public responses. If your invoices need one of them, add an explicit measurement export to your integration before relying on it.

| Operation and example | Internal quantity | Interpretation |
| --- | --- | --- |
| Score: a query is evaluated against two candidates | `pairs = 2` | This counts evaluated pairs; it does not replace their token measurements |
| Extract: Docling confirms three processed pages; another attempt has unavailable page metadata | `pages = 3`; the other attempt's fallback `0` needs reconciliation | Preserve conversion evidence so missing page metadata is not silently billed as zero work |
| Extract: audio has 1,601 source samples at 16,000 Hz | `audio_ms = 101` | `ceil(1601 / 16000 × 1000) = 101`. Wall-clock transcription time does not enter this calculation |
| Extract: LightOnOCR processes one image as one page | `pages = 1` | Its page meter takes precedence over a generic image count, including the SGLang profile configured with `meter_pages: true` |

#### Image-count limitation

The dedicated Florence-2, OWLv2, Grounding DINO, Donut, GLM-OCR, PaddleOCR-VL, and MinerU adapters process the first image of each item, while their inherited image-count hook counts all supplied images. With two images in one item, that hook can report two although only one was processed. Send one image per item on these profiles. The SGLang vision-extraction adapter overrides the hook to count the first image; its page-metered LightOnOCR profile uses `pages` instead. See the [default image hook](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/base.py), [Florence-2 extraction](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/florence2/__init__.py), and [SGLang override](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/sglang_vision_extract/adapter.py).

**SDK request metadata:** optional `request` metadata describes an entire request and may be repeated on its items. Count that request once when your deployment supplies it. Moving to a cluster does not make encode/extract SDK convenience results a universal export of worker units.

## Timing and resource cost

Source: [packages/sie_server/src/sie_server/core/timing.py](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/core/timing.py)

Timing fields explain latency. Depending on the path, these include `queue_ms`, `tokenization_ms`, `inference_ms`, `postprocessing_ms`, and `total_ms`. Generation can report `ttft_ms` and `tpot_ms`.

An inference duration is not a count of billable tokens or a GPU invoice. Concurrent requests can share a batch; loading, idle capacity, CPU work, and provider rounding have different cost implications. A 300-token answer can take different amounts of time on different hardware while retaining the same token quantity. Similarly, `audio_ms` measures input audio duration, even if transcription finishes faster than real time.

Use [monitoring and observability](/docs/deployment/monitoring/) for capacity and performance analysis. Aggregated telemetry does not provide a durable, customer-attributed transaction ledger by itself.

## Connect usage to your billing service

Your application chooses the billable dimensions and owns their prices. Fields such as `credits_charged` or `rate_book_version`, when supplied by a managed integration, are additional billing annotations and are not required outputs of a self-hosted engine. A token-priced generation service might calculate:

```text
amount = prompt_tokens × input_price_per_token
       + completion_tokens × output_price_per_token
```

Keep that calculation separate from measurement collection:

1. **Identify the work.** Assign your own customer and operation identifiers before dispatch. Retain the resolved model, profile, deployed SIE version, and attempt correlation where available.
2. **Record the outcome and quantities.** Preserve the reported units, their source, timestamps, and completion status. Keep unknown measurements distinguishable from measured zeros.
3. **Apply a versioned pricing policy.** Store which prices and rounding rule produced the amount. Decide explicitly how partial results, cancellations, and failed attempts are handled.
4. **Deliver billing events durably.** Deduplicate your records and retry delivery to your invoice system. Persisting an event once does not make a repeated inference request the same execution. A connection loss can leave the caller uncertain whether work finished; preserve that uncertainty for reconciliation.

Check your selected models and API paths with known inputs before enabling invoices. Include a long input that truncates, a multi-item request, a measured zero, an unavailable count, and an interrupted stream. An adapter returning a useful result does not guarantee every billing dimension is available on that route.

## Implementation references

These links point to the public `superlinked/sie` repository. Follow the corresponding tag or commit for your deployed version.

- [Token-count hooks](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/_base_adapter.py) and [metering tests](https://github.com/superlinked/sie/blob/main/packages/sie_server/tests/test_metering_units.py)
- [GLiNER document-token counting](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/gliner/__init__.py) and [Docling page counting](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/adapters/docling/adapter.py)
- [Audio duration calculation](https://github.com/superlinked/sie/blob/main/packages/sie_audio_prep/src/lib.rs) and [worker result units](https://github.com/superlinked/sie/blob/main/packages/sie_server/src/sie_server/queue_executor.py)
- [Standalone usage tests](https://github.com/superlinked/sie/blob/main/packages/sie_server/tests/api/test_usage_reporting.py), [gateway response construction](https://github.com/superlinked/sie/blob/main/packages/sie_gateway/src/handlers/proxy.rs), and [SDK result types](https://github.com/superlinked/sie/blob/main/packages/sie_sdk/src/sie_sdk/types.py)