Carve Off Tasks, Not Models

Short version: the open source conversation has collapsed into a race between very large model releases, and that race has almost nothing to do with the work most production systems actually do. Superlinked CEO Daniel Svonava went on The Joe Reis Show to argue for a different starting question. Which task? Define the task well enough to describe its input and output, and a 27B open model on one GPU will usually cover it at a fraction of the cost of the frontier.

At a glance

Open source attention pools around the biggest releases

Daniel has mixed feelings about the current moment. All the oxygen goes to whatever very large model shipped most recently, and the rotation is brutal: GLM-5.2 for a week, then Kimi K3 overnight, then a Qwen release the day after. Everyone is chasing marginal gains at the top of the leaderboard while the rest of the ecosystem, the encoders and rerankers and extractors and the 27B generalists, gets almost no coverage.

His framing of the problem is that the attention is very unevenly spread. The gap is not capability. It is that nobody writes the blog post about the model you can actually run.

Open weights no longer guarantee a low bill

The old story was simple. Open source meant cheap; proprietary meant expensive. Daniel’s read of Artificial Analysis is that the story has flipped for the biggest open releases: cost per task on a frontier-scale open model can land above a comparably configured proprietary model, because reasoning traces at maximum effort burn an enormous number of output tokens.

Joe pushed back with the practical version of this. He pays for a Claude Max subscription, an OpenAI subscription, and Gemini through Workspace. To self-host the headline open models he would need hardware he does not own, and to rent them he would be paying for something he already has. He called it the McDonald’s-at-home problem.

Both objections point the same way. If the open model you are comparing needs data-center hardware, its license is close to irrelevant to you; you are still renting someone else’s GPUs. Which is exactly why Superlinked builds for the models you can run on hardware you can get.

The orchestrator pattern works, and then the bill arrives

The pattern Daniel sees in engineering teams is now well established. A smart model sits on top as orchestrator, kicks off sub-agents for long-horizon work, watches their traces mid-run, and drops corrections into the queue without tearing down the turn. Sub-agents run for hours; the orchestrator runs for days.

Then two problems show up.

The first is mechanical. Long-running agents generate terabytes of logs and local state, and nothing cleans it up.

The second is the invoice. Daniel’s VP of engineering friends in San Francisco watched per-engineer spend go from a couple hundred dollars a month to low thousands. Sub-agents are the expensive part, so first-degree optimization means moving predictable work down a tier: the overnight security sweeps, the library migrations, the batch pipelines. Carve those off, put an eval behind them, and let the orchestrator keep the interactive work.

The unit of optimization is the task

This is the part worth taking away from the episode. Superlinked does not start from a model shortlist.

We don’t think about it as okay which open source model, but more like how do we carve off and define a task.

Once the task has a described input and a described output, the model becomes a shopping decision. Maybe an off-the-shelf open model covers it. Maybe you combine two, add a LoRA, optimize the prompt, or fine-tune on a few hundred examples. The output you want is predictable behavior at a small fraction of the cost, plus latency low enough to put the step in a user-facing path. Small models can answer in tens of milliseconds.

There is a second benefit that gets less airtime: a bounded output distribution. A very clever general model will occasionally surprise you, which is delightful in a chat window and expensive in a pipeline. A specialist model on a narrow task does the same thing every time.

A 27B open model matches last year’s frontier

Daniel’s working definition of a small model is refreshingly concrete. Take the GPUs that are affordable and easy to actually buy, then ask what runs on one of them without sharding. Today that lands somewhere around 27B to 35B parameters.

Qwen3.6-27B sits at roughly 96% of GPT-5.1 on the Artificial Analysis Intelligence Index. Think about what that means in practice. Every enterprise workload that was signed off on a late-2025 frontier model, and everything built on GPT-4o before that, now has a single-GPU open equivalent. Intercom’s Fin is the case study Daniel points to, and it is no longer unusual.

2024 was the year everyone talked about fine-tuning and nobody did it, because proprietary progress was too fast and the open base models were too weak. That arithmetic has changed. If your users are not asking your support agent to prove the Riemann hypothesis, and they are not, a 27B model with some task-specific work on top will hold the line.

Browse the model catalog to see what fits on the hardware you have.

A big context window is not the same as good coverage of it

Joe mentioned using a frontier model as a book editor across a 400-page manuscript, catching numeric inconsistencies between chapters. Daniel’s caveat: coverage inside a context window is uneven. A tiny model now ships with a very large window, and it will not connect distant dots across that window the way its headline number suggests. Context rot is the name people use for this; needle-in-a-haystack tests are the crude way of measuring it. Shoving everything in is not a retrieval strategy.

Text-to-SQL is a loop problem before it is a model problem

Joe raised a question he gets often: which model should you use for text-to-SQL? Daniel’s answer had two halves.

There are open models fine-tuned specifically to emit SQL, Cypher, or GraphQL, and they produce valid queries more reliably than a general model does. That is the easy half. The harder half is that one-shotting the query with a very large model is the wrong shape entirely.

What works is a fast model that pokes around. It runs a query, reads the result, revises its understanding, pulls in documentation and institutional knowledge (the column that used to be dollars and is now euros), and caches the artifacts it builds along the way so the next query starts warmer. Database queries are cheap and scalable. A small model that iterates twenty times beats a large model that guesses once, on cost, on latency, and often on correctness.

Follow the money to find the task worth moving

Daniel used to run Superlinked on milestones and a roadmap. Not anymore. The planning horizon has compressed to about a week; the Monday steering meeting feels quarterly and the daily standup feels weekly, while people ship ten PRs in between. Almost nothing on the board takes three weeks.

What replaces the roadmap is instrumentation, plus a willingness to let people run ahead and then read the bills afterwards.

Just follow the money. Oh okay, this thing over here is costing a third of our infra bill. What is that?

That question is how you find real work. When every agent rebuilds the same environment from scratch, you have a caching problem to go fix; when they all hammer the same query pattern, you have an index to add. Nobody planned either fix. The cost line surfaced them.

To read the traces at all, the team open sourced the tool they built for it. synty is a single binary that runs on developer machines and agent VMs, keeps every session and PR searchable locally, and optionally syncs to an S3 or GCS bucket so a team shares one memory. There is no central server to deploy; readers build the index themselves, and summaries come from Qwen3-0.6B on your own CPU. It found a bug where a Python environment kept cloning itself across a worktree until tens of gigabytes of junk accumulated on the machine. Nobody reads agent thinking traces by hand. An agent reading a connected dataset of traces, PRs, and issues does.

Daniel’s framing for all of this comes from Jobs to be Done, which he says he rereads roughly monthly because his engineering brain resists it. You do not want a drill bit; you want a hole in the wall. Engineers like to think about drill bits all day.

The tinkerers and the labs are converging

Asked what he wants from the next twelve months, Daniel gave two answers.

More diversity in where open models come from, Europe included. And a break in the manic focus on the largest models, so teams can start parceling out what their systems are actually doing. With frontier intelligence hitting diminishing returns, the interesting problem moves from “can this be solved” to “how do we bring the rest of the organization and the rest of the stack along.” Build systems break daily. CI has three sources of runners and none of them agree.

The older split was clean: proprietary professionals with unreachable hardware on one side, hobbyists doing basic things that did not quite work on the other. Those camps are collapsing into each other. Tinkerers got professional; the labs discovered their edge was thinner than advertised and started letting other models run inside their own agent tooling.

His closing point was about agency. Being subject to how somebody else wakes up in the morning is a bad position, even when that somebody is excellent at their job. If everything is one prompt away, agency is close to the only differentiator left.

Try the carved-off-task approach

Every model in the SIE catalog serves from one endpoint, whether you run it on a laptop or on a cluster in your own cloud. Generation needs the GPU generation image (or Apple MLX on port 8081); see the quickstart for the full local path.

docker run --gpus all -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cuda12-sglang

from sie_sdk import SIEClient

# On Apple Silicon: SIEClient("http://localhost:8081")
client = SIEClient("http://localhost:8080")

# A carved-off generation task on a small model.
result = client.generate(
    "Qwen/Qwen3-0.6B",
    "Reply with a single word: the capital of France.",
    max_new_tokens=16,
    temperature=0.0,
)
print(result["text"])  # Paris

The same code runs against a production cluster. SIE ships a load-balancing gateway, KEDA autoscaling down to zero, Grafana dashboards, and Terraform modules for GKE, EKS, and AKS. All Apache 2.0.

FAQ

What counts as a small model in 2026? The practical definition is a model that fits on one GPU you can readily buy, with no sharding. Today that ceiling sits around 27B to 35B parameters, and it moves as memory gets cheaper. Older cards count too: an L4 fleet two or three generations behind the frontier runs this class of model fine.

How do I decide which task to move off a frontier model? Start with the bill rather than the architecture diagram. Find the line item eating a disproportionate share, then check whether the work behind it has a describable input and output. Repetitive extraction, classification, embedding, reranking, and structured generation all qualify. Open-ended reasoning across ambiguous context usually does not.

Does a smaller model mean worse output? On a narrow task, often the opposite. A specialist model gives you a tighter output distribution, which is what you want inside a pipeline. The failure mode of a large general model is unpredictability, and that is expensive to debug.

Why doesn’t a one-million-token context window solve my retrieval problem? Because coverage inside the window is uneven. A model can accept a million tokens and still fail to connect two related facts sitting far apart in them. Retrieval that puts the right few thousand tokens in front of the model beats a large window filled indiscriminately.

Where can I watch the full episode? Daniel’s conversation with Joe Reis is on YouTube. Joe is the co-author of Fundamentals of Data Engineering and hosts The Joe Reis Show weekly.