LoRA Adapters
LoRA adds trained low-rank weights to a base model. SIE can keep several compatible adapters alongside that base and select one for a request. Support depends on the model adapter and its profile; adding a request field does not give an unsupported model LoRA support.
Declare adapters before selecting them
Section titled “Declare adapters before selecting them”A request may select only an adapter declared by its model profile. It cannot supply an arbitrary repository or filesystem path for the server to load.
For an existing Qwen/Qwen3-Embedding-4B model configuration using SGLangEmbeddingAdapter, add a profile like this. Replace the example repository with your compatible, trained LoRA:
profiles: lora: extends: default adapter_options: loadtime: lora_paths: legal: org/qwen3-legal-lora max_loras_per_batch: 2 preload_loras: []Here legal is the served name; org/qwen3-legal-lora is the source. A dictionary entry can also pin its source with id and a full 40-character commit SHA in revision. The existing base-model configuration and its revision remain in place.
For SGLang, an omitted preload_loras preloads all declarations. An empty list defers them; a list of declared names preloads that subset. Only declared adapters can load later.
Use lora_adapter in requests
Section titled “Use lora_adapter in requests”For native encode, extract and score calls, put lora_adapter in the runtime options. These examples select the lora profile above and its legal adapter.
from sie_sdk import SIEClientfrom sie_sdk.types import Item
with SIEClient("http://localhost:8080") as client: result = client.encode( "Qwen/Qwen3-Embedding-4B", Item(text="breach of fiduciary duty"), options={"profile": "lora", "lora_adapter": "legal"}, )import { SIEClient } from "@superlinked/sie-sdk";
const client = new SIEClient("http://localhost:8080");const result = await client.encode( "Qwen/Qwen3-Embedding-4B", { text: "breach of fiduciary duty" }, { adapterOptions: { profile: "lora", lora_adapter: "legal" } },);The equivalent native HTTP request is:
curl -X POST http://localhost:8080/v1/encode/Qwen/Qwen3-Embedding-4B \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -d '{"items":[{"text":"legal query"}],"params":{"options":{"profile":"lora","lora_adapter":"legal"}}}'The field location follows the endpoint’s existing request envelope:
| API | Selector location |
|---|---|
| Native encode or extract | params.options.lora_adapter |
| Native score | options.lora_adapter |
| Generate or chat completions | Top-level lora_adapter |
| System One or Decisions compatibility API | Top-level lora_adapter, a SIE extension |
The compatibility field still requires a model profile that supports LoRA. With an OpenAI-compatible client, a SIE extension can be supplied through extra_body without changing the upstream request schema.
For generation, use the Python SDK’s lora_adapter argument or the TypeScript SDK’s loraAdapter option. Both send the canonical top-level wire field. Generation retains its canonical-only interface.
Preserve defaults and reject conflicting selections
Section titled “Preserve defaults and reject conflicting selections”Omitting the native selector preserves the selected profile’s runtime default. Explicit null selects the base model, including when the profile has a LoRA default:
with SIEClient("http://localhost:8080") as client: base_result = client.encode( "Qwen/Qwen3-Embedding-4B", Item(text="general query"), options={"profile": "lora", "lora_adapter": None}, )TypeScript uses null in adapterOptions; leaving the property undefined omits it. Generation’s omitted and null selectors both select its base model.
Existing native and decision clients can keep using the legacy request aliases lora and lora_id. lora_adapter and lora accept a declared served name or source ID. The legacy lora_id remains restricted to declared source IDs, so legal in the example belongs in lora_adapter, while org/qwen3-legal-lora is a source ID.
If more than one selector is supplied, their values must be identical. A name and null conflict, even if one spelling is legacy. A served name and its source ID are also different values; send one selector. Conflicts and invalid types are rejected before loading or dispatch.
These request aliases do not rename model-configuration fields. Existing adapter_options.runtime.lora_id defaults remain supported where the model profile permits them.
Loading and capacity depend on the model adapter
Section titled “Loading and capacity depend on the model adapter”The registry can load declared LoRAs on demand for adapters that expose runtime loading. This includes supported PEFT adapters and SGLang generation and embedding adapters. SGLang is not limited to startup-only loading.
The explicit reranker and entity-extraction profiles support runtime selection:
- Standard, unquantized BERT and XLM-R rerankers use
sie_server.adapters.cross_encoder.runtime_lora:CrossEncoderLoraAdapteras theiradapter_path. - GLiNER2 entity extraction uses
sie_server.adapters.gliner2.entity_lora:GLiNER2EntityLoraAdapter. This profile is entity-only; it does not enable classification, relations or typed decisions.
Set the explicit adapter_path and declare compatible lora_paths in the profile.
A first request can encounter LORA_LOADING while its adapter loads. Python encode calls retry the documented loading response within a bounded budget; load failures and exhausted capacity are still errors. Fetching and attaching are separate operations, and attaching may require waiting for active inference. Loading is not a latency guarantee.
Resident limits, preload pins and available memory determine whether another adapter can load. The registry can evict an eligible least-recently-used adapter; pinned startup adapters are not eviction candidates. An evicted declaration can load again on a later request. Memory use depends on the trained adapter and backend, so measure it for your deployment.
Enable LoRA for native decision models
Section titled “Enable LoRA for native decision models”Decision models use separate, explicit LoRA profile classes. Choose the class for the native model and a bundle that includes it:
| Model family | Profile adapter_path | Bundle |
|---|---|---|
| Laya | sie_server.adapters.laya.runtime_lora:LayaLoraAdapter | default |
| GLiClass uni-encoder | sie_server.adapters.gliclass.runtime_lora:GLiClassLoraAdapter | default |
| GLiNER2.5 Decide | sie_server.adapters.gliner2.decide_lora:GLiNER2DecideLoraAdapter | transformers5 |
| Perplexity native decisions | sie_server.adapters.pplx_decider.runtime_lora:PerplexityDecisionLoraAdapter | transformers5 |
For example, add this profile to the existing convaiinnovations/laya configuration. Replace the source with a compatible LoRA trained for that base model:
profiles: lora: extends: default adapter_path: sie_server.adapters.laya.runtime_lora:LayaLoraAdapter adapter_options: loadtime: lora_paths: legal: org/laya-legal-loraSet SIE_MAX_LORAS_PER_MODEL=2 in the server environment to keep at most two LoRAs loaded per model (the default is 10). The serving engine applies this limit to every profile and overrides a profile’s max_loaded_loras load option.
Select it on a native extract call with options={"profile": "lora", "lora_adapter": "legal"}. The pinned {id, revision} declaration form works here too. These profiles load declared adapters on demand; they do not use SGLang’s preload_loras option.
Laya, GLiNER2.5 Decide and Perplexity retain their native typed-decision schemas, including when called through the System One or Decisions compatibility API. GLiClass retains its labels and label_groups classification interface; it does not accept a typed output_schema.
Use ordinary PEFT LoRA exports with task_type: null, bias: none and explicit target-module names. These profiles require unquantized base models and adapter_model.safetensors. They reject replacement heads (modules_to_save), embedding deltas, DoRA, RS-LoRA and per-layer rank or alpha patterns. Targets must match the native model’s supported linear layers; a LoRA trained for a different architecture is not interchangeable.
Ordinary profiles keep their existing execution path. In a LoRA-enabled profile, an adapted request runs eagerly and selecting the base restores its normal execution policy, including its configured CUDA graphs. The LoRA-enabled profile still performs selection and synchronization. Loading and attaching can also delay requests; hot loading does not promise zero overhead or watch a repository for new revisions.
Stage artifacts for offline loading
Section titled “Stage artifacts for offline loading”SGLang LoRA loaders, supported PEFT embedding loaders, and the explicit cross-encoder, GLiNER2 entity and decision profiles above use SIE’s configured cache policy. Set SIE_LOCAL_CACHE to the local Hugging Face-style cache; SIE_CLUSTER_CACHE can provide a shared copy. Set SIE_HF_FALLBACK=false to disable their Hub fallback. Loading fails if the required adapter is unavailable.
Stage both adapter_config.json and adapter_model.safetensors for the declared revision. A partial snapshot is not a cache hit, and an explicit commit pin must match exactly. Without a revision pin, the loader follows the cached refs/main commit rather than choosing another available snapshot.
Hub-backed base checkpoints for Laya, GLiNER2.5 Decide and Perplexity also use SIE_LOCAL_CACHE. For these native loaders, either SIE_HF_FALLBACK=false or HF_HUB_OFFLINE=1 disables Hub access during checkpoint resolution.
An offline deployment also needs its required base-model and tokenizer artifacts staged, along with installed runtime dependencies. Disabling Hub fallback does not supply missing files or dependencies.
For more configuration details, see Profiles and the Model Catalog.