---
title: How to generate text with SIE
description: The generate primitive runs text generation on small LLMs you host yourself, with the same client and routing as encode, score, and extract.
canonical_url: https://superlinked.com/docs/generate
last_updated: 2026-06-26
---

**The `generate` primitive runs text generation on small LLMs you host yourself.** Same client, same routing, and the same model catalog as the other primitives, so a generation model sits alongside your encoders and rerankers instead of behind a separate service.

> **Note — Preview:**
>
> `generate` is an early surface. The blocking call returns the full result once generation finishes; for incremental output, stream tokens with `stream_generate` (see [Streaming](#streaming)). For chat-shaped requests, use the OpenAI-compatible `/v1/chat/completions` endpoint instead.

## Basic Usage

Pass a model, a prompt, and a token budget. `max_new_tokens` is required.

#### Python

```python
from sie_sdk import SIEClient

client = SIEClient("http://localhost:8080")

result = client.generate(
    "Qwen/Qwen3-4B-Instruct-2507",
    "Write a one-sentence summary of vector search.",
    max_new_tokens=128,
)
print(result["text"])
```

#### TypeScript

```typescript
import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");

const result = await client.generate(
  "Qwen/Qwen3-4B-Instruct-2507",
  "Write a one-sentence summary of vector search.",
  { maxNewTokens: 128 },
);
console.log(result.text);

await client.close();
```

## Common Options

| Option | Type | Description |
|--------|------|-------------|
| `max_new_tokens` | int | Hard cap on output tokens. Required. |
| `temperature` | float | Sampling temperature. Defaults to `1.0`. |
| `top_p` | float | Nucleus sampling cutoff. Defaults to `1.0`. |
| `stop` | list of strings | Stop sequences that end generation early. |

The result carries the generated `text`, a `finish_reason`, token `usage`, and SIE-native timing (`ttft_ms`, `tpot_ms`). See the [SDK Reference](/docs/reference/sdk/) for the full surface.

## Streaming

Stream tokens as they arrive with `stream_generate`. It yields chunks, each carrying a `text_delta`; the terminal chunk (`done`) also carries `usage` and `ttft_ms`.

#### Python

```python
for chunk in client.stream_generate(
    "Qwen/Qwen3-4B-Instruct-2507",
    "Write a haiku about vector search.",
    max_new_tokens=64,
):
    print(chunk.get("text_delta", ""), end="", flush=True)
    if chunk.get("done"):
        print(f"\nttft_ms={chunk.get('ttft_ms')}")
```

#### TypeScript

```typescript
for await (const chunk of client.streamGenerate(
  "Qwen/Qwen3-4B-Instruct-2507",
  "Write a haiku about vector search.",
  { maxNewTokens: 64 },
)) {
  process.stdout.write(chunk.text_delta);
  if (chunk.done) console.log(`\nTTFT: ${chunk.ttft_ms}ms`);
}
```

## Models

Generation models are listed in the [Model Catalog](/models) alongside the encoders and rerankers. `Qwen/Qwen3-4B-Instruct-2507` is a solid default; `Qwen/Qwen3-0.6B` is a smaller, faster option for short prompts.

On a multi-pool cluster, generation models may run on a dedicated GPU pool. Target it with the `gpu` parameter, for example `gpu="rtx6000"`, so the request lands on a pool that serves the generation bundle.

## What's Next

- [Model Catalog](/models) - all supported models
- [SDK Reference](/docs/reference/sdk/) - full client API
- [API Reference](/docs/reference/api/) - HTTP endpoints