Skip to content
Most AI tasks no longer need a frontier model Read launch post

How to use tool calling with SIE

Tool calling lets a model hosted on SIE ask your code to run a function, such as a weather lookup or a database query, and then answer with the result. It works on the OpenAI-compatible POST /v1/chat/completions endpoint, so the official OpenAI SDKs work with a base URL change.

SIE does not run your tools. The model only returns a request to call one, with the arguments it picked; your program executes the function and sends the result back. Every tool-calling exchange follows the same loop:

  1. You send the conversation plus a list of tool definitions.
  2. The model answers with one or more tool_calls instead of text, and finish_reason is "tool_calls".
  3. Your code runs each requested function locally.
  4. You append the results as tool messages and call the endpoint again. The model reads them and writes its answer, or asks for more calls.

The examples below use Qwen/Qwen3.5-4B against a local SIE at http://localhost:8080. Install the OpenAI client first: pip install openai for Python, npm install openai for TypeScript.

A tool is a function name and a JSON Schema for its arguments, plus a description. The model reads all of it, so write the description for the model: say what the function returns and when to call it.

import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, for example Paris"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city"],
},
},
}
]
response = client.chat.completions.create(
model="Qwen/Qwen3.5-4B",
messages=[{"role": "user", "content": "What's the weather in Paris right now?"}],
tools=tools,
max_tokens=512,
temperature=0,
)
choice = response.choices[0]
print("finish_reason:", choice.finish_reason)
for call in choice.message.tool_calls or []:
print(call.id, call.function.name, json.loads(call.function.arguments))

The assistant message carries tool_calls and a null content. Each call has an id and type: "function", plus a function object holding name and arguments. The arguments are a JSON string, not an object, so parse them before use. Abridged:

{
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_9f2c41d07a6b4e1c8d3e5a72",
"type": "function",
"function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
}
]
},
"finish_reason": "tool_calls"
}
]
}

The model can also answer in plain text without calling anything. Then tool_calls is absent and finish_reason is "stop", as in any other chat completion.

Tool definitions are rendered into the prompt by the model’s chat template. They count as prompt tokens and take room in the context window, so a long list of verbose tools costs you on every request.

Return the Result and Get the Final Answer

Section titled “Return the Result and Get the Final Answer”

To finish the exchange, append two things to messages: the assistant message with its tool_calls, unchanged, and one {"role": "tool", "tool_call_id": ..., "content": ...} message per call. The tool_call_id must match the call’s id. Then call the endpoint again with the same tools.

The scripts below run the whole loop with a stub get_weather that returns fixed data. They stop after five rounds so a model that keeps calling tools cannot loop forever. They also set temperature to 0: with tool_choice left on auto and default sampling, a small model now and then answers “I have no real-time data” instead of calling the tool.

import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
MODEL = "Qwen/Qwen3.5-4B"
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, for example Paris"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city"],
},
},
}
]
def get_weather(city: str, unit: str = "celsius") -> dict:
"""Stub: returns fixed data instead of calling a weather service."""
if not isinstance(city, str) or unit not in ("celsius", "fahrenheit"):
raise ValueError("city must be a string and unit must be celsius or fahrenheit")
temperature = 18 if unit == "celsius" else 64
return {"city": city, "temperature": temperature, "unit": unit, "conditions": "partly cloudy"}
LOCAL_FUNCTIONS = {"get_weather": get_weather}
def run_tool(name: str, arguments: str) -> dict:
function = LOCAL_FUNCTIONS.get(name)
if function is None:
return {"error": f"unknown tool: {name}"}
try:
return function(**json.loads(arguments))
except (TypeError, ValueError) as exc:
return {"error": f"bad arguments for {name}: {exc}"}
messages = [{"role": "user", "content": "What's the weather in Paris right now? Answer in one sentence."}]
for _ in range(5):
response = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=tools,
max_tokens=512,
temperature=0,
)
message = response.choices[0].message
if not message.tool_calls:
print(message.content)
break
messages.append(
{
"role": "assistant",
"content": message.content,
"tool_calls": [
{
"id": call.id,
"type": "function",
"function": {"name": call.function.name, "arguments": call.function.arguments},
}
for call in message.tool_calls
],
}
)
for call in message.tool_calls:
result = run_tool(call.function.name, call.function.arguments)
print(f"ran {call.function.name}({call.function.arguments}) -> {result}")
messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
else:
print("Stopped after 5 rounds without a final answer.")

tool_choice decides whether the model has to call a tool. It requires tools in the same request.

tool_choiceWhat the model does
"auto" (default)Decides for itself whether to call tools, and how many.
"none"Answers in text and calls nothing, even though tools are declared.
"required"Must call at least one of the declared tools.
{"type": "function", "function": {"name": "get_weather"}}Must call that one function.

"required" and the named form are enforced with constrained decoding. That uses the same machinery as response_format, so a forced tool_choice cannot be combined with response_format in one request; "auto" and "none" can.

Limits on the tools array:

  • At most 64 tools per request, and at least one when the field is present.
  • Each entry is {"type": "function", "function": {...}}. Function names must match ^[a-zA-Z0-9_-]{1,64}$.
  • parameters is a JSON Schema and is checked against the same depth and size caps as a response_format schema.
  • The legacy OpenAI functions and function_call fields are rejected. Use tools and tool_choice.

A model can return more than one call in a single assistant message, for example when one question needs the weather in two cities. parallel_tool_calls defaults to true, which allows that. The round-trip scripts above already handle it: they run every call in tool_calls and append one tool message per call, each with its own tool_call_id.

Set parallel_tool_calls to false when your code can handle only one call per turn. The setting is a guard; it does not steer the model toward a single call. The request below asks about two cities, so the model starts a second call and the request fails:

curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen3.5-4B",
"max_tokens": 512,
"parallel_tool_calls": false,
"messages": [{"role": "user", "content": "Compare the weather in Paris and in Rome."}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}},
"required": ["city"]
}
}
}]
}'

The cluster gateway answers 500 with code parallel_tool_calls_violated. SIE does not quietly drop the extra call, because a dropped call would leave you with half an answer and no sign of it. Ask for one thing per turn, or leave the setting on and run every call. The single-node server on Apple Silicon does not forward parallel_tool_calls to its backend. It checks the response instead: a second call fails the request with a 502, or with an error event on a stream.

With stream: true, tool calls arrive in delta.tool_calls. Each entry has an index that identifies the call. The first chunk for a call carries its id and function.name; the function.arguments follow.

SIE emits each call’s arguments in one piece once the model closes the call, not token by token, so a long argument shows up after a pause rather than as a trickle. Write the client to concatenate argument fragments by index anyway. That is what the OpenAI contract allows, and the code below works either way.

import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, for example Paris"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city"],
},
},
}
]
stream = client.chat.completions.create(
model="Qwen/Qwen3.5-4B",
messages=[{"role": "user", "content": "What's the weather in Paris and in Rome?"}],
tools=tools,
max_tokens=512,
temperature=0,
stream=True,
)
calls = {}
for chunk in stream:
if not chunk.choices:
continue
choice = chunk.choices[0]
delta = choice.delta
if delta.content:
print(delta.content, end="", flush=True)
for part in delta.tool_calls or []:
call = calls.setdefault(part.index, {"id": None, "name": None, "arguments": ""})
if part.id:
call["id"] = part.id
if part.function and part.function.name:
call["name"] = part.function.name
if part.function and part.function.arguments:
call["arguments"] += part.function.arguments
if choice.finish_reason:
print(f"\nfinish_reason: {choice.finish_reason}")
for index, call in sorted(calls.items()):
print(index, call["id"], call["name"], json.loads(call["arguments"]))

The SIE clients expose the same endpoint: chat_completions() in Python and chatCompletions() in TypeScript accept the same tool fields and return the OpenAI-shaped response. Use them when you already have an SIEClient and want its retry handling while a model starts.

from sie_sdk import SIEClient
client = SIEClient("http://localhost:8080")
result = client.chat_completions(
"Qwen/Qwen3.5-4B",
[{"role": "user", "content": "What's the weather in Paris right now?"}],
tools=[
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}
],
max_tokens=512,
)
print(result["choices"][0]["message"].get("tool_calls"))

Support is per model. A model accepts tools only when its config sets capabilities.tools: true; any other model answers 400 with code unsupported_field. GET /v1/models reports the flag for each entry of its models array, under capabilities.tools:

curl -s http://localhost:8080/v1/models

The flag is on the models entries only. The OpenAI-shaped data list in the same response carries ids and nothing else, so client.models.list() in the OpenAI SDK does not show it.

Models with tool support include Qwen/Qwen3.5-4B, Qwen/Qwen3.8-27B-FP8, zai-org/GLM-5.3-Flash, and google/gemma-4-31B-it. The Model Catalog lists the rest of the generation models.

Check the context window too. The default profile of Qwen/Qwen3.5-4B allows 8,192 tokens for prompt and output together, which an agent with dozens of tools and a long history fills fast. Pick a larger-window profile by adding its name to the model string: Qwen/Qwen3.5-4B:long-context serves the same weights with a 32,768-token window.

CauseWhat you get
The model does not support tools400, code unsupported_field, param tools
A field the endpoint does not accept, including functions and function_call400, code unsupported_field, param set to the field name
More than 64 tools, or a tool whose name breaks ^[a-zA-Z0-9_-]{1,64}$ or whose parameters exceed the schema caps400, code invalid_request, param pointing at the entry, for example tools[0].function.name
tool_choice without tools, or a forced tool_choice together with response_format400, code invalid_request, param tool_choice
A tool message without tool_call_id400, code invalid_request, param messages[N].tool_call_id
"required" or a named tool_choice on the single-node server on Apple Silicon400, code unsupported_field; that backend cannot force a call
max_tokens runs out in the middle of a tool callThe call is never returned. Code MODEL_OUTPUT_PARSE_ERROR (500 on a non-streamed request, an error event on a stream). Raise max_tokens.
The model writes a tool call SIE cannot parseCode MODEL_OUTPUT_PARSE_ERROR, as above
The model makes a second call with parallel_tool_calls: falseCode parallel_tool_calls_violated (500, or an error event on a stream)

The codes are the ones the cluster gateway returns. A single-node sie-server answers the first three rows with 400 as well, but some of its code strings differ, so branch on the status and param.

Coding agents run this same loop hundreds of times per task, with many tools declared on every turn. SIE also serves the Anthropic Messages API at POST /v1/messages, where tools come back as tool_use blocks and you return tool_result blocks; it translates each request onto /v1/chat/completions, so the same models and limits apply. For agent setup, follow the runnable examples for OpenCode and Claude Code.

Talk to us

Tell us about your use case and we'll get back to you shortly.