---
title: Small Open Models for Legal Work
description: How to break a legal job into steps, have small open models find the personal data and the clauses and code apply the playbook, and hand the negotiation rounds to an agent whose answers code still checks.
canonical_url: https://superlinked.com/blog/legal-work-on-small-open-models
last_updated: 2026-11-11
---

[Legal work](/legal) runs on reading, at law firms and in-house legal teams alike. Litigation teams review documents in discovery and research the law behind every argument, deal teams read through a target's contracts in due diligence, in-house lawyers review and negotiate the contracts the business signs, and compliance teams track what regulators publish. Partners and senior counsel decide, but associates and paralegals do most of the reading, page by page, against a protocol, a playbook or a research memo that someone already wrote. AI tools for lawyers have gone after every one of those jobs. The decisions all that reading feeds are the ones a firm gets judged on: what to hand the other side, which of their wording to accept, whether an authority really supports the sentence that cites it.

In this post I'll start with why that reading is risky and what the big legal AI teams already run. Then I'll go through how I think about a legal job: break it into steps, decide which steps run as a fixed workflow and which need an agent, and work out which steps code, a small model, a bigger model or the lawyer should do. After that come three worked examples from the [SIE cookbook](https://github.com/superlinked/cookbook/tree/main/legal): redacting a discovery set, checking an NDA against a playbook (with an agent for the negotiation rounds that follow), and cite-checking a brief. I'll finish with how to look at your own legal work the same way and how to run the recipes yourself.

## Why the Reading Is Risky

None of this work is glamorous, and all of it is risky in a quiet way. A production can run to thousands of documents, and every one coded "produce with redactions" gets read for the same handful of things the ESI protocol lists. An NDA gets read against the same playbook positions as the last forty, and a brief's citations get checked against the same research folder, one by one. It's the same few things read over and over, which is exactly where people slip, and one missed detail costs far more than the hour it took. A disclosed SSN of someone who isn't even in the case is personal data the firm has to ask the other side to return. A five-year confidentiality period that slipped through on the eighth NDA of the week gets found years later, when someone wants to hire from the counterparty. And a quote with one word changed won't show up in a search of the case it cites.

Client data is also confidential and sometimes privileged, and bar regulators have started saying where it may go (the [legal page](/legal) has the details). That's a question about where your models run, and it's worth answering before anyone asks about accuracy.

## What Legal AI Teams Already Run

Four of the big legal AI teams have said publicly what they run:

- **Harvey** [partnered with Engram](https://www.harvey.ai/blog/post-training-update-harvey-tenet) to train a Qwen3.8-27B model that, in their August 2026 research preview, "completes tasks competitively with leading frontier models while reducing cost per query by 90%".
- **Thomson Reuters** [launched Thomson](https://thomsonreuters.com/en/press-releases/2026/august/thomson-reuters-leverages-its-world-class-data-assets-to-launch-its-own-frontier-model), its own LLM, which "starts from a strong open-source foundation", and CoCounsel's [Tabular Analysis runs on it by default](https://www.thomsonreuters.com/en-us/posts/innovation/cocounsel-legal-august-2026-releases/).
- **LexisNexis**, as [VentureBeat reported](https://venturebeat.com/ai/small-models-as-paralegals-lexisnexis-distills-models-to-build-ai-assistant), mostly relies on a fine-tuned Mistral model. Its CTO Jeff Reihl put it plainly: "You don't need GPT-4o to do the assessment of a query, so we use it for more sophisticated work, and we switch models out."
- **Latham & Watkins** [bought its own Nvidia GPU servers](https://www.lw.com/admin/upload/SiteAttachments/Latham-Buys-Nvidia-Servers-to-Set-Up-In-House-AI-Systems.pdf) (Financial Times, September 2026) and is fine-tuning Nvidia Nemotron 3 open-weight models in-house. Its CIO, Rene Mendoza: "we don't want to put it to any cloud vendor".

Reihl's line is about cost: a small model to assess the query and a large one for the more sophisticated work. Harvey's 27B sits in between, an open mid-size model (on the verge of large) tuned to take over work that went to frontier models. Latham's reason is control: it wants a model it can tune on its own work and run where its client data is allowed to go.

<BlogSieCta />

## How I Break a Legal Job Into Steps

I start by writing down the steps a paralegal or associate takes, in order. Then I decide how they run. If I can write them down before the first document arrives and they're the same every time, it's a workflow: Python runs the steps and calls a small model where one needs reading. If the steps depend on what someone says next, that part goes to an agent, and I look for whatever code can still check in its answers. I've written about [when to use a workflow, an agent or a mix of the two](/blog/agentic-vs-workflows-small-models) separately, so here I'll show what it looks like on legal work.

Then, step by step, I ask who should do it. A lot of what a lawyer checks is already structured somewhere (the review platform's export, the matter's lawyer list, the ESI protocol's redaction list, the CLM intake record, the citator), and code reads those. Finding one kind of thing, like "find the text that looks like personal data" or "point at the governing-law clause", goes to the smallest model trained for it, one that returns something my code can check: a span (the exact characters, with their position) or a score. "Does this passage actually support the sentence citing it?" has no rule I could write, so that goes to a bigger 27B model, on a short list the small models picked. And privilege calls, redlines, accepting the other side's wording and signing stay with the lawyer.

The document a model reads was often written by the other side, so a quote that matches it only shows the quote is faithful to that document, and nothing more. So I keep two kinds of check apart: quotes get checked against the original document, and claims about policy or authorization get checked against records the firm controls, like a list the firm keeps or labels the model never sees.

The recipes below use models as they ship, and that works because a rule a lawyer would sign makes every decision. When no rule can (say, deciding whether a contract term is unfair), I'd first measure the models as they ship on your own labeled reviews. If they fall short, and a simple classifier trained on those labels doesn't reach your target either, that's when a model fine-tuned on your past reviews pays off, and Superlinked's [study on a public set of unfair Terms of Service clauses](/legal) (LexGLUE) shows what that buys you. There's more on that in [off the shelf or fine-tuned](/blog/small-open-models-across-industries#off-the-shelf-or-fine-tuned).

## Three Legal Examples, Worked Through

All three recipes run their models as shipped, with the decisions in `rules.yaml`, on synthetic data. They show how I'd break each job into steps; they aren't a product.

### Redacting a Discovery Set Before It Goes Out

As a first example, take a litigation paralegal redacting a discovery set. They open a document coded "produce with redactions", find a crew member's Social Security number pasted into an insurance email, draw a box over it, write a line in the redaction log, and move on to the wire confirmation where the same account number appears twice, once as `4471900321` and once as `4471 9003 21`. The paralegal follows the protocol's list every time, so the recipe, [redact_documents_with_log](https://github.com/superlinked/cookbook/tree/main/legal/redact_documents_with_log), runs it as a workflow.

![Flowchart of the redaction: pii-redact finds spans in the document text, a value without a label nearby or a failed call goes to a person, Python redacts every spelling, sweeps the redacted text and checks paragraphs for lawyers, and the associate decides the flags from the text, log and spans](/blog/legal-work-on-small-open-models/redaction-flow.webp)

One small model, [`superlinked/pii-redact`](/models/superlinked-pii-redact), finds candidate spans with character offsets, so the code knows exactly what to redact and which line to log. In an injury report it marked "3 March 1991" as a date of birth and the delivery date in the same sentence as a plain date. It also tagged the word "SSN" itself as a Social Security number, which is very thorough and not very helpful. That's why Python decides what gets redacted: it counts a span only if a label sits on the same line just before it, anything the model finds without a label stays readable and goes to a person instead of out the door, and a number redacted once gets redacted in every spelling across the set, which is how `4471 9003 21` was caught with no help from the model. Privilege uses no model at all; a paragraph to, from or about a lawyer on the matter list gets flagged, and the associate decides every flag.

The cases no rule can settle still go to a person. The recipe can't tell an insurer's claim reference shaped like an SSN (`404-18-2210`) from a real one, and a date of birth in a table, with "DOB" up in the column header, has its label on another line.

### Checking an NDA Against Your Playbook

For a second example, take an in-house lawyer reviewing NDAs. They open the eighth counterparty NDA of the week and read it top to bottom against the playbook. The recipe, [review_contract_against_playbook](https://github.com/superlinked/cookbook/tree/main/legal/review_contract_against_playbook), checks five positions (term, governing law, non-solicit, residuals and permitted disclosures), and since that first review is the same every time, it's a workflow.

![Flowchart of the NDA review: docling reads the Word file, Python checks every heading came back and both parties are named, gliner2-large-v1 points at each playbook position, Python checks the evidence and compares with the playbook, and each position comes out matches_position, deviates with the standard wording, unclear or not_found for the lawyer](/blog/legal-work-on-small-open-models/nda-flow.webp)

[`docling`](/models/docling) converts each Word file, and [`fastino/gliner2-large-v1`](/models/fastino-gliner2-large-v1) points at the clause for each playbook position. Python compares each clause with the playbook. One test NDA, from Corrin Fabrication, runs two years in Section 4, and Section 7 says confidentiality survives five years after it ends; the extractor only had to find the two numbers, and Python added them and flagged seven years against a three-year limit. Python also ignored "Delaware", which the extractor marked in a preamble, because a company's state of incorporation is not its choice of law, however confidently it's marked. Every departure comes to the lawyer with the standard wording ready to paste and the name of whoever has to approve an exception.

### The NDA Negotiation Rounds That Follow

The workflow is done once it writes its review, but the negotiation has only started. Over one to four rounds of email, the other side keeps sending something new: they reject a redline, so the lawyer asks what the playbook lets them offer instead; their counsel comes back with new wording, so the lawyer wants to know whether it meets the playbook, and then wants the reply drafted. Each question depends on what the other side just said and on the last answer, so nobody can write those steps down in advance, and that part goes to an agent with the lawyer at the screen. The agent can look up the intake record, run the workflow's review, read a playbook position and its fallback (a second position legal has already approved), and check new wording. The cookbook tests three of these conversations, one each with Corrin Fabrication, Lowfield Ridge and Orlaith Marine, and runs each one five times.

Code still checks each answer, in two places. The new wording never gets checked on its own: a tool swaps it into the full NDA and reruns the workflow's checks on the whole draft. That matters on Corrin. Their counsel offered to cut Section 7 to "survive for one (1) year after this Agreement expires or is terminated", which is fine only if you know the agreement runs two years, and that's in another section. In the full NDA that's two years plus one, three in total, which is within the limit. Checked on its own, the clause says "one year after the agreement ends" with no idea how long the agreement runs, so the workflow could only call it unclear. Then, after every answer, code checks what the agent wrote. Any number of years or place it mentions has to come from the playbook, the NDA, a tool result or the lawyer. When it says a clause matches the playbook or deviates from it, that has to be what the latest check actually returned. It can only agree to wording that passed. And the draft reply has to mention every point the review left open.

On Lowfield Ridge, the lawyer asked for the fallback (four years without approval on this kind of deal), and the other side came back with four years "and, for any trade secret, for so long as it remains a trade secret." That tail means trade secrets stay confidential with no end date, and the lawyer said no to it. The code passed it in every run anyway, because its rules only look for a number of years and have nothing for "for so long as". The agent pointed out the tail itself in all five runs ("The tool passed it, but I wouldn't rely on that yet"), but the code is blind to it, and that's why the lawyer still reads every answer.

On Orlaith Marine, the lawyer said "The demo is Thursday. Just draft the reply accepting Singapore." There's no fallback for governing law, so the agent didn't draft that reply. In most runs it declined, and when the lawyer then said "Fine, draft the note to General Counsel instead", it drafted the note quoting the clause word for word. In two runs it skipped ahead and wrote the note straight away. One of those then just replied that the note was done. That's true, but the check only counts a draft written in answer to the latest request, so that run was scored as producing no draft. Over five runs of each of the three conversations, 14 of 15 matched the expected outcome, and that count includes the five Lowfield runs where the code passed the trade-secret tail, because showing that gap was the point of the example. The agent added $0.05 to $0.09 per conversation on top of the workflow's calls.

### Cite-Checking a Brief, the One Step That Needs a Bigger Model

The third example is a cite check. The night before a brief is filed, a cite checker goes through its citations one by one. In the recipe, [check_citations_against_sources](https://github.com/superlinked/cookbook/tree/main/legal/check_citations_against_sources), most of that is lookups and string checks: the citator, the jurisdiction and the draft's quoted words, all in Python before any model runs.

![Flowchart of the cite check: Python pulls each citation and its sentence, a citation missing from the research folder is not_in_folder, Python checks the citator, jurisdiction and quotes, Qwen3-Reranker-4B ranks every paragraph, Qwen3.8-27B-FP8 picks the passage with a quote, Python checks the quote and the part of the opinion, and the result is support_found or check_citation for the associate to fix](/blog/legal-work-on-small-open-models/cite-check-flow.webp)

[`Qwen/Qwen3-Reranker-4B`](/models/qwen-qwen3-reranker-4b) narrows each authority to its closest paragraphs, and only then does [`Qwen/Qwen3.8-27B-FP8`](/models/qwen-qwen3-8-27b-fp8) read those few and say which one supports the sentence, with a quote. Python accepts the quote only if it's word for word in the passage the model named, and checks which part of the opinion that passage came from.

On one test authority, Pell v. Drayton, that last check is what got it right. The reranker scored a dissent 0.9998, and the 27B model said it supports the brief, with a quote that really is in the passage. The paragraph starts with "Judge Varga, dissenting." The model read right past it. Python didn't, because every passage carries the part of the opinion it came from, a label the model never sees. Python also caught a quote where "necessary" had become "reasonably necessary".

![Pell v. Drayton: the reranker scored the dissent 0.9998 and the 27B model said it supports the sentence with an exact quote, while Python's part check saw the paragraph is from the dissent and marked check_citation](/blog/legal-work-on-small-open-models/pell-dissent.webp)

Even when everything passes, the associate reads it before the partner signs. The checks prove the quote is real and from the majority; whether it carries the argument is still for a lawyer to decide.

## Where Your Own Legal Work Fits

Look for the reading your team does against something already written down: a protocol, a playbook, a research folder. The cookbook READMEs name a few more: public versions of sealed filings, medical records before production, a customer's DPA against your positions, a supplier MSA's liability cap, an expert report's quotes against its exhibits. Run the checks you can write down as a workflow, give the follow-up conversation to an agent whose answers your code checks, and keep the signature with the lawyer.

## Run It Yourself

The easiest way to see how this feels is to run a recipe on [SIE Serverless](/serverless) with the synthetic documents that ship with it. You need [uv](https://docs.astral.sh/uv/) and an [SIE API key](https://console.superlinked.com/console/keys):

```bash
git clone https://github.com/superlinked/cookbook
cd cookbook
cp .env.example .env    # then set SIE_API_KEY
cd legal/redact_documents_with_log
uv run recipe.py
```

For the NDA agent, `cd legal/review_contract_against_playbook`, add an `ANTHROPIC_API_KEY` to `.env`, and run `uv run --group agent agent.py`; with `--chat --request NDA-2026-0447` you can play the lawyer yourself. Serverless sends your inputs to Superlinked, which is fine for synthetic documents. Whether client data may go to a hosted service is your firm's rule, and when it can't, point `SIE_BASE_URL` at an SIE server in your own environment; the code stays the same, and [SELF_HOSTING.md](https://github.com/superlinked/cookbook/blob/main/SELF_HOSTING.md) has the setup. That keeps the task-model calls in your environment, but the NDA agent still calls the Anthropic API, so run it only with data approved for that service.

## Conclusion

Most legal reading is done against a rule someone already wrote, and you can take any of those jobs apart the same way: write down the steps, run the ones you know as a workflow with small models finding things and code applying the protocol or the playbook, give the back-and-forth with the other side to an agent whose answers code still checks, and leave the decisions (what to hand over, which wording to accept) and the signature with the lawyer.

If you try one of these on documents you've already reviewed by hand, I'd love to hear which step broke first. Until next time.