How to use tool calling with SIE
Tool calling lets a model hosted on SIE ask your code to run a function, such as a weather lookup or a database query, and then answer with the result. It works on the OpenAI-compatible POST /v1/chat/completions endpoint, so the official OpenAI SDKs work with a base URL change.
SIE does not run your tools. The model only returns a request to call one, with the arguments it picked; your program executes the function and sends the result back. Every tool-calling exchange follows the same loop:
- You send the conversation plus a list of tool definitions.
- The model answers with one or more
tool_callsinstead of text, andfinish_reasonis"tool_calls". - Your code runs each requested function locally.
- You append the results as
toolmessages and call the endpoint again. The model reads them and writes its answer, or asks for more calls.
The examples below use Qwen/Qwen3.5-4B against a local SIE at http://localhost:8080. Install the OpenAI client first: pip install openai for Python, npm install openai for TypeScript.
Define a Tool and Get a Tool Call
Section titled “Define a Tool and Get a Tool Call”A tool is a function name and a JSON Schema for its arguments, plus a description. The model reads all of it, so write the description for the model: say what the function returns and when to call it.
import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": {"type": "string", "description": "City name, for example Paris"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, }, "required": ["city"], }, }, }]
response = client.chat.completions.create( model="Qwen/Qwen3.5-4B", messages=[{"role": "user", "content": "What's the weather in Paris right now?"}], tools=tools, max_tokens=512, temperature=0,)
choice = response.choices[0]print("finish_reason:", choice.finish_reason)for call in choice.message.tool_calls or []: print(call.id, call.function.name, json.loads(call.function.arguments))import OpenAI from "openai";
const client = new OpenAI({ baseURL: "http://localhost:8080/v1", apiKey: "not-needed" });
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [ { type: "function", function: { name: "get_weather", description: "Get the current weather for a city.", parameters: { type: "object", properties: { city: { type: "string", description: "City name, for example Paris" }, unit: { type: "string", enum: ["celsius", "fahrenheit"] }, }, required: ["city"], }, }, },];
const response = await client.chat.completions.create({ model: "Qwen/Qwen3.5-4B", messages: [{ role: "user", content: "What's the weather in Paris right now?" }], tools, max_tokens: 512, temperature: 0,});
const choice = response.choices[0];console.log("finish_reason:", choice.finish_reason);for (const call of choice.message.tool_calls ?? []) { if (call.type !== "function") continue; console.log(call.id, call.function.name, JSON.parse(call.function.arguments));}curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Qwen/Qwen3.5-4B", "max_tokens": 512, "temperature": 0, "messages": [{"role": "user", "content": "What'\''s the weather in Paris right now?"}], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": {"type": "string", "description": "City name, for example Paris"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["city"] } } }] }'The assistant message carries tool_calls and a null content. Each call has an id and type: "function", plus a function object holding name and arguments. The arguments are a JSON string, not an object, so parse them before use. Abridged:
{ "choices": [ { "index": 0, "message": { "role": "assistant", "content": null, "tool_calls": [ { "id": "call_9f2c41d07a6b4e1c8d3e5a72", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"} } ] }, "finish_reason": "tool_calls" } ]}The model can also answer in plain text without calling anything. Then tool_calls is absent and finish_reason is "stop", as in any other chat completion.
Tool definitions are rendered into the prompt by the model’s chat template. They count as prompt tokens and take room in the context window, so a long list of verbose tools costs you on every request.
Return the Result and Get the Final Answer
Section titled “Return the Result and Get the Final Answer”To finish the exchange, append two things to messages: the assistant message with its tool_calls, unchanged, and one {"role": "tool", "tool_call_id": ..., "content": ...} message per call. The tool_call_id must match the call’s id. Then call the endpoint again with the same tools.
The scripts below run the whole loop with a stub get_weather that returns fixed data. They stop after five rounds so a model that keeps calling tools cannot loop forever. They also set temperature to 0: with tool_choice left on auto and default sampling, a small model now and then answers “I have no real-time data” instead of calling the tool.
import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")MODEL = "Qwen/Qwen3.5-4B"
tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": {"type": "string", "description": "City name, for example Paris"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, }, "required": ["city"], }, }, }]
def get_weather(city: str, unit: str = "celsius") -> dict: """Stub: returns fixed data instead of calling a weather service.""" if not isinstance(city, str) or unit not in ("celsius", "fahrenheit"): raise ValueError("city must be a string and unit must be celsius or fahrenheit") temperature = 18 if unit == "celsius" else 64 return {"city": city, "temperature": temperature, "unit": unit, "conditions": "partly cloudy"}
LOCAL_FUNCTIONS = {"get_weather": get_weather}
def run_tool(name: str, arguments: str) -> dict: function = LOCAL_FUNCTIONS.get(name) if function is None: return {"error": f"unknown tool: {name}"} try: return function(**json.loads(arguments)) except (TypeError, ValueError) as exc: return {"error": f"bad arguments for {name}: {exc}"}
messages = [{"role": "user", "content": "What's the weather in Paris right now? Answer in one sentence."}]
for _ in range(5): response = client.chat.completions.create( model=MODEL, messages=messages, tools=tools, max_tokens=512, temperature=0, ) message = response.choices[0].message if not message.tool_calls: print(message.content) break
messages.append( { "role": "assistant", "content": message.content, "tool_calls": [ { "id": call.id, "type": "function", "function": {"name": call.function.name, "arguments": call.function.arguments}, } for call in message.tool_calls ], } ) for call in message.tool_calls: result = run_tool(call.function.name, call.function.arguments) print(f"ran {call.function.name}({call.function.arguments}) -> {result}") messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})else: print("Stopped after 5 rounds without a final answer.")import OpenAI from "openai";
const client = new OpenAI({ baseURL: "http://localhost:8080/v1", apiKey: "not-needed" });const MODEL = "Qwen/Qwen3.5-4B";
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [ { type: "function", function: { name: "get_weather", description: "Get the current weather for a city.", parameters: { type: "object", properties: { city: { type: "string", description: "City name, for example Paris" }, unit: { type: "string", enum: ["celsius", "fahrenheit"] }, }, required: ["city"], }, }, },];
// Stub: returns fixed data instead of calling a weather service.function getWeather(args: { city: string; unit?: string }) { const unit = args.unit ?? "celsius"; return { city: args.city, temperature: unit === "celsius" ? 18 : 64, unit, conditions: "partly cloudy" };}
function runTool(name: string, rawArguments: string): unknown { if (name !== "get_weather") return { error: `unknown tool: ${name}` }; try { const args = JSON.parse(rawArguments); if (typeof args?.city !== "string") return { error: "get_weather needs a string 'city'" }; if (args.unit !== undefined && args.unit !== "celsius" && args.unit !== "fahrenheit") { return { error: "unit must be celsius or fahrenheit" }; } return getWeather(args); } catch (err) { return { error: `bad arguments for ${name}: ${String(err)}` }; }}
const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [ { role: "user", content: "What's the weather in Paris right now? Answer in one sentence." },];
let answered = false;for (let round = 0; round < 5 && !answered; round++) { const response = await client.chat.completions.create({ model: MODEL, messages, tools, max_tokens: 512, temperature: 0, }); const message = response.choices[0].message; const calls: { id: string; name: string; arguments: string }[] = []; for (const call of message.tool_calls ?? []) { if (call.type === "function") { calls.push({ id: call.id, name: call.function.name, arguments: call.function.arguments }); } } if (calls.length === 0) { console.log(message.content); answered = true; break; }
messages.push({ role: "assistant", content: message.content, tool_calls: calls.map((call) => ({ id: call.id, type: "function" as const, function: { name: call.name, arguments: call.arguments }, })), }); for (const call of calls) { const result = runTool(call.name, call.arguments); console.log(`ran ${call.name}(${call.arguments}) ->`, result); messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) }); }}if (!answered) console.log("Stopped after 5 rounds without a final answer.");# Second request of the loop. Use the id from the first response's tool call;# any id works as long as the assistant message and the tool message agree.curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Qwen/Qwen3.5-4B", "max_tokens": 512, "messages": [ {"role": "user", "content": "What'\''s the weather in Paris right now? Answer in one sentence."}, {"role": "assistant", "content": null, "tool_calls": [{ "id": "call_9f2c41d07a6b4e1c8d3e5a72", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"} }]}, {"role": "tool", "tool_call_id": "call_9f2c41d07a6b4e1c8d3e5a72", "content": "{\"city\": \"Paris\", \"temperature\": 18, \"unit\": \"celsius\", \"conditions\": \"partly cloudy\"}"} ], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": {"type": "string", "description": "City name, for example Paris"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["city"] } } }] }'Control Tool Use with tool_choice
Section titled “Control Tool Use with tool_choice”tool_choice decides whether the model has to call a tool. It requires tools in the same request.
tool_choice | What the model does |
|---|---|
"auto" (default) | Decides for itself whether to call tools, and how many. |
"none" | Answers in text and calls nothing, even though tools are declared. |
"required" | Must call at least one of the declared tools. |
{"type": "function", "function": {"name": "get_weather"}} | Must call that one function. |
"required" and the named form are enforced with constrained decoding. That uses the same machinery as response_format, so a forced tool_choice cannot be combined with response_format in one request; "auto" and "none" can.
Limits on the tools array:
- At most 64 tools per request, and at least one when the field is present.
- Each entry is
{"type": "function", "function": {...}}. Function names must match^[a-zA-Z0-9_-]{1,64}$. parametersis a JSON Schema and is checked against the same depth and size caps as aresponse_formatschema.- The legacy OpenAI
functionsandfunction_callfields are rejected. Usetoolsandtool_choice.
Several Calls in One Turn
Section titled “Several Calls in One Turn”A model can return more than one call in a single assistant message, for example when one question needs the weather in two cities. parallel_tool_calls defaults to true, which allows that. The round-trip scripts above already handle it: they run every call in tool_calls and append one tool message per call, each with its own tool_call_id.
Set parallel_tool_calls to false when your code can handle only one call per turn. The setting is a guard; it does not steer the model toward a single call. The request below asks about two cities, so the model starts a second call and the request fails:
curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Qwen/Qwen3.5-4B", "max_tokens": 512, "parallel_tool_calls": false, "messages": [{"role": "user", "content": "Compare the weather in Paris and in Rome."}], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"] } } }] }'The cluster gateway answers 500 with code parallel_tool_calls_violated. SIE does not quietly drop the extra call, because a dropped call would leave you with half an answer and no sign of it. Ask for one thing per turn, or leave the setting on and run every call. The single-node server on Apple Silicon does not forward parallel_tool_calls to its backend. It checks the response instead: a second call fails the request with a 502, or with an error event on a stream.
Streaming Tool Calls
Section titled “Streaming Tool Calls”With stream: true, tool calls arrive in delta.tool_calls. Each entry has an index that identifies the call. The first chunk for a call carries its id and function.name; the function.arguments follow.
SIE emits each call’s arguments in one piece once the model closes the call, not token by token, so a long argument shows up after a pause rather than as a trickle. Write the client to concatenate argument fragments by index anyway. That is what the OpenAI contract allows, and the code below works either way.
import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": {"type": "string", "description": "City name, for example Paris"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, }, "required": ["city"], }, }, }]
stream = client.chat.completions.create( model="Qwen/Qwen3.5-4B", messages=[{"role": "user", "content": "What's the weather in Paris and in Rome?"}], tools=tools, max_tokens=512, temperature=0, stream=True,)
calls = {}for chunk in stream: if not chunk.choices: continue choice = chunk.choices[0] delta = choice.delta if delta.content: print(delta.content, end="", flush=True) for part in delta.tool_calls or []: call = calls.setdefault(part.index, {"id": None, "name": None, "arguments": ""}) if part.id: call["id"] = part.id if part.function and part.function.name: call["name"] = part.function.name if part.function and part.function.arguments: call["arguments"] += part.function.arguments if choice.finish_reason: print(f"\nfinish_reason: {choice.finish_reason}")
for index, call in sorted(calls.items()): print(index, call["id"], call["name"], json.loads(call["arguments"]))Use the SIE SDK
Section titled “Use the SIE SDK”The SIE clients expose the same endpoint: chat_completions() in Python and chatCompletions() in TypeScript accept the same tool fields and return the OpenAI-shaped response. Use them when you already have an SIEClient and want its retry handling while a model starts.
from sie_sdk import SIEClient
client = SIEClient("http://localhost:8080")
result = client.chat_completions( "Qwen/Qwen3.5-4B", [{"role": "user", "content": "What's the weather in Paris right now?"}], tools=[ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, } ], max_tokens=512,)print(result["choices"][0]["message"].get("tool_calls"))import { SIEClient } from "@superlinked/sie-sdk";
const client = new SIEClient("http://localhost:8080");
const reply = await client.chatCompletions({ model: "Qwen/Qwen3.5-4B", messages: [{ role: "user", content: "What's the weather in Paris right now?" }], tools: [ { type: "function", function: { name: "get_weather", description: "Get the current weather for a city.", parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"], }, }, }, ], max_tokens: 512,});console.log(reply.choices[0]?.message.tool_calls);
await client.close();Which Models Support Tools
Section titled “Which Models Support Tools”Support is per model. A model accepts tools only when its config sets capabilities.tools: true; any other model answers 400 with code unsupported_field. GET /v1/models reports the flag for each entry of its models array, under capabilities.tools:
curl -s http://localhost:8080/v1/modelsThe flag is on the models entries only. The OpenAI-shaped data list in the same response carries ids and nothing else, so client.models.list() in the OpenAI SDK does not show it.
Models with tool support include Qwen/Qwen3.5-4B, Qwen/Qwen3.8-27B-FP8, zai-org/GLM-5.3-Flash, and google/gemma-4-31B-it. The Model Catalog lists the rest of the generation models.
Check the context window too. The default profile of Qwen/Qwen3.5-4B allows 8,192 tokens for prompt and output together, which an agent with dozens of tools and a long history fills fast. Pick a larger-window profile by adding its name to the model string: Qwen/Qwen3.5-4B:long-context serves the same weights with a 32,768-token window.
Errors and Edge Cases
Section titled “Errors and Edge Cases”| Cause | What you get |
|---|---|
| The model does not support tools | 400, code unsupported_field, param tools |
A field the endpoint does not accept, including functions and function_call | 400, code unsupported_field, param set to the field name |
More than 64 tools, or a tool whose name breaks ^[a-zA-Z0-9_-]{1,64}$ or whose parameters exceed the schema caps | 400, code invalid_request, param pointing at the entry, for example tools[0].function.name |
tool_choice without tools, or a forced tool_choice together with response_format | 400, code invalid_request, param tool_choice |
A tool message without tool_call_id | 400, code invalid_request, param messages[N].tool_call_id |
"required" or a named tool_choice on the single-node server on Apple Silicon | 400, code unsupported_field; that backend cannot force a call |
max_tokens runs out in the middle of a tool call | The call is never returned. Code MODEL_OUTPUT_PARSE_ERROR (500 on a non-streamed request, an error event on a stream). Raise max_tokens. |
| The model writes a tool call SIE cannot parse | Code MODEL_OUTPUT_PARSE_ERROR, as above |
The model makes a second call with parallel_tool_calls: false | Code parallel_tool_calls_violated (500, or an error event on a stream) |
The codes are the ones the cluster gateway returns. A single-node sie-server answers the first three rows with 400 as well, but some of its code strings differ, so branch on the status and param.
Coding Agents
Section titled “Coding Agents”Coding agents run this same loop hundreds of times per task, with many tools declared on every turn. SIE also serves the Anthropic Messages API at POST /v1/messages, where tools come back as tool_use blocks and you return tool_result blocks; it translates each request onto /v1/chat/completions, so the same models and limits apply. For agent setup, follow the runnable examples for OpenCode and Claude Code.
What’s Next
Section titled “What’s Next”- Generate text - the native
generateprimitive - OpenAI compatibility - the other OpenAI-shaped endpoints
- API Reference - HTTP endpoints