LFM3: A Technical Deep Dive into Liquid AI's Tri-Hybrid 2B Multimodal Family

TL;DR — Liquid AI released LFM3 on October 27, 2026, a new generation of Liquid Foundation Models built for "specialized intelligence." The first release is six 2B-class open-weight models — text, vision, audio, omni, and two decision models — all branched from a single LFM3-1.7B-Base backbone trained on 22T tokens. The headline change is a tri-hybrid architecture: of 24 layers, 13 are gated short convolutions (kernel size 7, up from 3), 5 are sliding-window attention (256-token window), and 6 are global attention, with head-wise gated attention, partial RoPE, and sandwich norm — every one of those details verified line-by-line against the released config.json and reference implementation in Section 2. LFM3-1.7B beats its predecessor LFM2.5-2.6B using 35% fewer parameters, averages 49.37 across its reported suite (vs 47.24 for Qwen3.5-9B), posts the best agent scores in its class (LiveClaw 45.60, AutomationBench 41.29, WebArena-Verified 38.42), carries a 256K-token context window, fits in ~1 GB at 4-bit, and decodes 1.8–2.8× faster than the next-fastest model on every edge device Liquid measured. Post-training runs through four stages ending in agentic RL inside real harnesses (Hermes Agent, Pi, OpenClaw). Weights are open under the LFM Open License v1.0 on Hugging Face. This post breaks down the architecture, the full benchmark record, each family member, the training pipeline, how to run it locally, and what it means for the edge-model race.

---

1. What LFM3 Actually Is

LFM3 is Liquid AI's third-generation Liquid Foundation Model family, announced October 27, 2026. The framing Liquid uses is "specialized intelligence": instead of one large general-purpose model in the cloud, LFM3 targets small models that can be fine-tuned on your own data and deployed where your application actually runs — a phone, a laptop, a Raspberry Pi, a car, or a single GPU box. The first LFM3 release covers the 2B parameter range, and every model in it is branched from the same LFM3-1.7B-Base checkpoint^[1].

That "one backbone, many specialists" structure is the core product decision. LFM2 and LFM2.5 shipped separate text, vision, and audio lines that evolved at different cadences. LFM3 collapses text, vision, audio, omni, and decision capabilities onto one shared tri-hybrid backbone, so a fine-tune, a quantization recipe, or a runtime improvement applies across the whole family. Liquid's claim is that LFM3 is "far more expressive than LFM2 while maintaining its efficiency" — the expressivity coming from architecture and a much larger pre-training run (22T tokens, versus roughly 15T for LFM2), the efficiency coming from keeping convolutions as the majority operator^[1].

The base and post-trained models are open-weight and available today on Hugging Face, the Liquid AI Playground, and the Liquid API^[1].

1.1 The Spec Sheet at a Glance

1.2 The Family: Six Models, One Backbone

The first LFM3 release includes six post-trained models plus the base checkpoint. Four cover different input modalities; two are decision models specialized from the vision and omni models^[1].

The models Liquid positions as replacements tell you where the product line is going: d1-2B replaces d1-3B with a third fewer parameters and higher scores, and LFM3-VL-2B succeeds the LFM2.5-VL series at a smaller size class^[1].

1.3 Key Capabilities

• Agent-first post-training: the text model is post-trained to run inside real agent harnesses — Hermes Agent, Pi, and OpenClaw — with a 256K-token context window and XML-format tool calls^[2].
• Tool search at scale: deferred tool loading keeps tool definitions out of the prompt until a search_tools call returns them, so a small model can operate over catalogs of thousands of tools without blowing the context budget^[2].
• Edge speed: at Q4_0 in llama.cpp with 32K context, LFM3-1.7B prefills at least 2.5× faster and decodes at least 1.8× faster than every compared model on all three measured devices, while using the least memory^[2].
• Cheap long context: a full 128K-token context needs 1.6 GB of KV cache, against 4.3 GB for Qwen3.5-4B and 5.6 GB for MiniCPM5-2B^[1].
• Made to be specialized: every model in the family can be fine-tuned on a single GPU, with TRL notebooks for SFT, DPO, GRPO, and continued pre-training^[2].

---

2. Architecture: The Tri-Hybrid Backbone, From the Config Up

LFM2 made Liquid's name with a hybrid of gated short convolutions and attention that ran fast on-device. LFM3 keeps that DNA but adds a third operator to the mix — and changes several details of the attention itself. The original version of this section paraphrased the launch post; it has since been rewritten to quote what Liquid has actually shipped. Everything below is cross-checked against four primary sources: the launch announcement^[1], the released config.json^[11], the Apache-2.0 reference implementation (modeling_lfm3.py)^[12], and — for the operator lineage — the LFM2 Technical Report^[13]. Layer counts, head counts, rotary settings, and normalization structure are quoted from the config and the code, not inferred; parameter totals and cache sizes are our derivations from those numbers, cross-checked against Liquid's published figures, and labeled as such.

Figure 1 — LFM3-1.7B block anatomy. Left: the verbatim 24-layer schedule from layer_types in config.json. Right: the sandwich-norm data flow of one decoder layer, the internals of both sequence mixers, and the parameter/state budgets derived in Sections 2.7–2.8.

2.1 The Exact 24-Layer Schedule

The config ships the layout as an explicit layer_types array, so there is no ambiguity about the ordering. Reading it out: the stack opens with two convolutions, then repeats a four-layer unit — global attention, conv, SWA, conv — five times (layers 3–22), and closes with a final global-attention + conv pair (layers 23–24). The counts land at 13 convolutions, 5 sliding-window layers, and 6 global attention layers. Note the discipline of the pattern: no two attention layers are ever adjacent. Every attention layer — local or global — is braced by convolutions, which is the structural reason conv work dominates the decode loop while attention provides the state.

The three-way mix is the headline, but the proportions are the actual design statement. Convolutions make up over half the stack — that is what keeps decode fast and memory bounded. The new ingredient relative to LFM2 is SWA: Liquid reports trying swaps of convolutions for other sequence blocks and finding sliding-window attention gave the best quality–speed tradeoff. The five SWA layers add a cheap "local attention" tier between the convolutions and the six global layers, and that intermediate tier is where the extra capability per parameter comes from. Per parameter, LFM3-1.7B matches LFM2-1.2B on short prompts across edge devices, and with 8K tokens in context it is about 25% faster^[1][14].

2.2 The Gated Short Convolution, Formally

The conv block is unchanged in structure from LFM2 — the LFM2 Technical Report gives its formal definition, and the LFM3 reference implementation instantiates it with the new kernel size^[13][12]. One bias-free input projection maps the 2,048-wide hidden state to 3 × 2,048 channels, split into three matrices B, C, x; the output is gated twice — once on the input side, once on the output side of a single causal depthwise convolution:

(B, C, x) = W_in h          # W_in ∈ ℝ^(2048 × 6144), no bias
u         = B ⊙ x           # input gate
z_t       = Σ_{i=0..6} K_i · u_{t−6+i}   # causal depthwise conv, kernel 7
y         = C ⊙ z           # output gate
out       = W_out y          # W_out ∈ ℝ^(2048 × 2048), no bias

Four implementation details from the code are worth knowing because they affect real deployments. First, the convolution is depthwise — one 7-tap filter per channel, 14,336 weights per layer — so its cost is linear in width and its per-layer parameter count is dominated by the two projections, not the filter. Second, during token-by-token decode the conv layer keeps a rolling state of just 6 positions × 2,048 channels (kernel − 1 frames), about 24 KB per layer in BF16 — this is why the 13 conv layers contribute almost nothing to long-context memory (Section 2.8). Third, inference uses dedicated width-7 Triton kernels (lfm3_causal_conv1d / lfm3_causal_conv1d_update) because the commonly bundled causal_conv1d kernels only accept widths 2–4 — an early sign of llama.cpp/Transformers builds needing LFM3-specific support. Fourth — and this is the genuinely novel bit — the convolution is document-aware: when a batch of packed documents is prefilled, a seq_idx-driven segmented convolution ensures that a conv window never reads across a document boundary. A transcript boundary in a packed agent prefill behaves like a hard reset, not a smoothing window; for a model whose target workload is prefilling thousands of tool-call traces, that distinction matters for both training throughput and correctness^[12].

Increasing the kernel from LFM2's 3 to 7 triples the conv receptive field per layer. Thirteen layers at kernel 7 give every position direct access to a 79-token backward window (13 × 6 + 1), before any attention is consulted — the conv stack alone handles phrase- and sentence-scale locality, freeing the attention tiers for longer-range structure. Liquid's own framing is that the search picked the kernel "up from 3 in LFM2" as part of the tri-hybrid redesign^[1].

2.3 Attention: GQA 4:1, QK-Norm, and the Head Gate

The config specifies 16 query heads and 4 key/value heads at head_dim 128 — grouped-query attention with a 4:1 ratio, the same trick that slashes KV traffic in every modern small model (Section 2.8 shows what that alone is worth at 128K context). Three refinements sit on top, and all three are visible in the reference code^[12]:

QK-RMSNorm. Query and key vectors are RMS-normalized per head (eps 1e-5) after their projections and before RoPE is applied. This bounds the attention-logit scale independently of training step and layer depth — a stabilizer carried over from LFM2, whose technical report notes its attention blocks use "RoPE with QK-Norm."^[13]

The head-wise sigmoid gate. This is LFM3's headline attention change. After the softmax-weighted sum over values, each head's output slice is multiplied by its own scalar gate before the output projection:

gate  = sigmoid(W_gate h)        # W_gate ∈ ℝ^(2048 × 16), no bias — one scalar per head
out   = W_o (attn ⊙ gate)        # applied per head, before flattening heads

Two specifics from the code deserve emphasis because the launch post doesn't spell them out. The gate is computed from the block's input hidden state (the post-operator-norm residual stream), not from the attention output — so it acts as a context-dependent per-head gain set by "what this layer received," and its cost is one 2,048 → 16 linear map, a trivial 32,768 parameters per attention layer. And it is applied on the output side, after attention weights have mixed values — so it re-weights heads, it does not sparsify attention patterns. Liquid cites two benefits: suppressing the activation outliers that hurt quantization (a big deal for a model whose flagship deployment is Q4_0 on phones), and improving training stability. It continues the gating theme of the LFM line — LFM2 gated its convolutions, LFM3 gates its attention heads too^[1][12].

No biases anywhere. Every projection in the block — Q, K, V, O, gate, both conv projections, and all three FFN matrices — is bias-free, matching conv_bias: false in the config. On edge runtimes that removes a whole class of small, awkwardly-shaped tensor ops from the decode graph.

2.4 RoPE: Per-Type Thetas and the Zero-Tail Partial Rotation

LFM3 gives each attention type its own rotary-embedding module with its own theta — the config's rope_parameters block separates full_attention from sliding_attention, a per-layer-type arrangement the code comments compare to Gemma 3. The settings are asymmetric in an interesting way:

The implementation detail that makes this more than a standard partial-RoPE toggle is what the code calls zero-tail partial rotation. The naive way to rotate half a head is to build 32 frequency pairs and rotate a 64-wide slice; Gemma-style implementations shrink the rotated dimension that way. LFM3 instead computes the inverse-frequency table over the full 128-dimension progression — keeping the frequency spacing a full-RoPE head would have — and then zeroes the tail 64 frequencies, leaving those dimensions unrotated. The rotated half therefore has exactly the frequency plan of a full-dimension RoPE head (which matters for anyone doing context-extension math or checkpoint surgery on those layers), while the unrotated half becomes a pure position-independent content channel. In the six global layers, the highest-frequency half rotates and the low-frequency half matches on content regardless of distance — the same family of "let some dimensions attend by content, not position" tricks recent long-context models use, and it pairs with the mid-training context extension described in Section 5^[12][11].

One honest observation ties this to the benchmark table in Section 3: RULER 128K is the one long-context score where LFM3-1.7B clearly trails Qwen3.5's full-attention models (65.56 vs 84.63/87.11). A zero-tail design halves the position-carrying capacity of each global head in exchange for a stable content channel, and five of the eleven attention layers see only a 256-token window. That is a plausible — unproven, but plausible — mechanical account of the retrieval gap, and we flag it as a hypothesis in Section 8.

2.5 Sandwich Norm — Four RMSNorms per Layer

The launch post's "sandwich norm: each block normalizes both its input and its output" understates what the code does. Reading Lfm3DecoderLayer.forward, each of the 24 layers runs this exact sequence:

residual = h
h1 = RMSNorm_pre(h)               # operator_norm
h1 = Mixer(h1)                    # conv (k=7) or attention
h1 = RMSNorm_post(h1)             # post_operator_norm   ← the "sandwich" top slice
h  = h1 + residual

f1 = RMSNorm_pre(h)               # ffn_norm
f1 = SwiGLU_FFN(f1)
f1 = RMSNorm_post(f1)             # post_ffn_norm        ← second sandwich
h  = f1 + h

That is four RMSNorms per layer — a pre-norm and a post-norm around each of the two sublayers — plus a final RMSNorm (named embedding_norm, but applied after the last layer, serving as the pre-LM-head norm) and no norm on the embedding input side. Pre-norm transformers normalize only the input; the post-norm pass keeps activations well-scaled through 24 layers of mixed operators, which matters when conv, SWA, and global branches have different activation magnitudes. The cost is real but modest — norms are 2,048-wide vectors, roughly 0.2 M parameters in total — and the benefit is precisely the one a quantization-first edge model needs: bounded activation ranges at every residual junction, in a network where LFM2 got away with pre-norm only^[12][11].

2.6 The FFN: SwiGLU at 3.75×, in Every Layer

Both the conv layers and the attention layers are followed by the same feed-forward network: a standard SwiGLU — w2(silu(w1(x)) ⊙ w3(x)) — projecting 2,048 → 7,680 → 2,048 with no biases. The 3.75× expansion ratio is in line with small-model practice, and the FFN is where the parameters actually live: 47.2 M per layer × 24 layers ≈ 1,133 M parameters, about 66% of the model. That proportion is worth internalizing when reading the fine-tuning claims in Section 6 — LoRA-style adaptation on the FFN projections touches two-thirds of the parameter mass, which is consistent with Liquid's single-GPU fine-tunes producing such large task-specific jumps^[12][11].

2.7 Where the 1.73 Billion Parameters Go

The model card states 1.73B parameters. The released config lets you rebuild that number exactly, which is the kind of cross-check a config-level analysis should make (derivation ours; total confirmed by the model card^[2][11]):

Three things fall out of this table. The attention layers — the most storied part of the architecture — hold barely 7% of the parameters; the FFN channel-mixers hold two-thirds. Tied embeddings mean the 128K vocabulary costs one embedding table, not two (262 M parameters, 15% of the model, doing double duty as the output head). And the 13 convolutions, for all their importance to speed, cost less than the embedding table — efficiency here is not "fewer layers," it is "cheap sequence mixing, expensive channel mixing."

2.8 The Memory Math Behind "1.6 GB at 128K"

Liquid's most-quoted edge number — a full 128K-token context in 1.6 GB of KV cache — is not a mystery; it follows from the config (derivation ours, total matches Liquid's published figure^[1][11]). Only the six global layers carry unbounded caches. Each of them stores, per token, K and V for 4 KV heads × 128 dims:

per token, per global layer:  2 (K+V) × 4 heads × 128 dims × 2 bytes (BF16) = 2,048 B
six global layers:            6 × 2,048 B = 12,288 B ≈ 12 KB/token
at 131,072 tokens:            131,072 × 12,288 B ≈ 1.61 GB   ← Liquid's "1.6 GB"
at 262,144 tokens:            ≈ 3.22 GB

The other nineteen layers are where the design pays off. The five SWA layers cache only their 256-token windows: 5 × 256 × 1,024 values × 2 B ≈ 2.6 MB, flat, regardless of context length. The thirteen conv layers hold rolling states of 6 frames × 2,048 channels: 13 × 24.6 KB ≈ 0.32 MB, flat. So a 256K-context session on a phone costs ~3.2 GB of cache, a 128K session ~1.6 GB, and a 32K session ~0.4 GB — against 4.3 GB and 5.6 GB that two 2B-4B-class competitors need at 128K. For comparison, the Q4_0 weights themselves are roughly 0.9–1.0 GB (1.73 B parameters at ~4 bits plus the BF16 norms and embedding overhead — our estimate), so at 128K the cache is still the larger half of the footprint, and the GQA 4:1 ratio is the single biggest lever that made it so.

2.9 Vocabulary and Special Tokens

LFM3 uses the same 128,000-token vocabulary as LFM2.5-8B-A1B, built to tokenize non-Latin scripts more efficiently. For a family that ships 16 languages — and whose audio model handles 18 — the tokenizer choice directly affects how much real content fits in the 256K window. The config also pins the control tokens: BOS 124,894, EOS 124,900, pad 124,893 — all clustered near the top of the vocabulary range, with tie_word_embeddings: true confirming the input/output weight sharing noted in Section 2.7^[11][1].

2.10 Hardware-in-the-Loop Search: What the Fitness Function Actually Was

The meta-story of LFM3's architecture is the search that produced it, and the LFM2 Technical Report documents that machinery in formal detail^[13]. The search space is wide: local-context candidates included gated short convolutions at various kernel sizes, sliding-window attention, linear-attention variants, and state-space models (S4, Liquid-S4, S5, RTF, Mamba, Mamba2), plus Liquid-Time-Constant networks; global-context candidates were GQA blocks at various group counts and head dimensions; position-wise blocks were SwiGLU FFNs with search-chosen expansion ratios; and the layout search decided the interleaving. Candidates were evaluated under explicit device budgets — time-to-first-token, p50/p95 decode latency at batch 1, prefill throughput, and peak RSS at 4K and 32K contexts — and ranked by hypervolume improvement on the quality–latency–memory Pareto frontier. Convolutions did not win by ideology; they won by surviving that gauntlet.

For LFM3, Liquid expanded the fitness hardware from "CPUs in one phone and one laptop" to 14 device and backend combinations: Samsung Galaxy S26 Ultra and OnePlus 15 phones, a Raspberry Pi 5, Apple M4 Max and AMD Ryzen AI laptops, Arm and x86 cloud CPUs, GPUs, and the Qualcomm Hexagon NPU — and re-ran the redesign with agentic long-context workloads as the target. Liquid is effectively productizing the search: the claims that matter to edge developers (decode speed at Q4_0, memory footprint, prefill at 32K) are the fitness function, not an afterthought^[1][14].

---

3. Benchmarks: LFM3-1.7B vs the Small-Model Field

Liquid's summary: "LFM3-1.7B can power agentic applications that used to need 9B parameters." The comparison set is LFM2.5-2.6B (its own predecessor), MiniCPM5-2B, Gemma 4 E4B, Qwen3.5-4B, and Qwen3.5-9B — with the 9B model included as the "several times larger" reference point. All models were served with vLLM in BF16 at full context length (262,144 tokens for LFM3-1.7B, Qwen3.5-4B, and Qwen3.5-9B; 131,072 for the others); evals default to temperature 0.6, top_p 0.95, and the Average row is the unweighted mean of every score in the table^[2].

Figure 2 — LFM3-1.7B vs LFM2.5-2.6B, MiniCPM5-2B, Qwen3.5-4B, and Qwen3.5-9B on six headline benchmarks. Source: Liquid AI LFM3-1.7B model card (October 2026).

3.1 Full Benchmark Table (Liquid-reported)

3.2 Where LFM3-1.7B Leads

• Agents, by a wide margin. LiveClaw 45.60 is 3.1× LFM2.5-2.6B (14.70) and 10 points clear of Qwen3.5-9B (35.77). WebArena-Verified 38.42 beats Qwen3.5-9B (28.94) — with LFM2.5-2.6B at just 3.02. AutomationBench 41.29 is 2.7× the 9B model's 15.10.
• Tool use. BFCL V4 (71.39) and ToolSandbox (80.36) lead the table outright.
• Instruction following, across all four IF benchmarks (IFEval 96.66, IFBench 73.03, IFStruct 93.72, Multi-IF 89.10) — IFBench by a 12.5-point margin over the runner-up.
• Non-hallucination. AA-Omniscience non-hallucination of 92.93 towers over the field (next best: 84.44), and the AA-Omniscience composite of +0.50 is the only non-negative score in the table — Liquid's small models are trained to decline rather than invent.
• Everyday coding. LiveCodeBench v6 73.46 leads the table, including the 9B model.
• The average. 49.37 across the whole suite — ahead of Qwen3.5-9B (47.24) at roughly one-fifth the parameters, and 17.8 points above LFM2.5-2.6B with 35% fewer parameters^[1][2].

3.3 Where It Still Trails

• Agentic coding. SWE-bench Verified (OpenCode): 43.40 vs Qwen3.5-9B's 51.20; Terminal-Bench 2.1: 12.92 vs 17.98. Liquid's model card says it outright: LFM3-1.7B is "not recommended for agentic coding or complex multi-step reasoning due to its small size."^[2]
• Knowledge-heavy tasks. GPQA Diamond 63.37 vs Qwen3.5-9B's 80.40; Humanity's Last Exam 7.17 (no tools) vs 16.41. Liquid's own guidance: use retrieval or search for knowledge-heavy tasks — with tools, LFM3's HLE score nearly doubles to 13.90.
• Long-context retrieval. RULER 128K is 65.56 vs Qwen3.5-4B's 84.63 and Qwen3.5-9B's 87.11. The 256K window is cheap to run (Section 6), but Qwen's full-attention models retrieve better at 128K. AA-LCR (52.83) is much closer to the pack.
• Fine-grained MCP tool selection. MCP-Atlas 32.57 trails both Qwen3.5 models (41.37, 42.67) — the one tool-use benchmark where it isn't first.

3.4 Independent Signals: MacPaw's macOS Agent Test

3.5 How to Read These Numbers

Three cautions before you extrapolate. First, this is a vendor-reported table on a benchmark suite that includes several agent environments closely tied to Liquid's own tooling (OpenClaw, Claw-Eval); treat the agentic margins as directional until independent replications land. Second, the suite's composition matters: the Average row weights 29 rows in which instruction-following and agentic tasks dominate, which flatters a model tuned precisely for those. Third, the comparison models were run at their full context lengths in BF16 — nobody's RULER 128K score reflects what you'd see at Q4_0 on a phone. The edge numbers in Section 6 are the ones to quote for on-device decisions.

---

4. The Rest of the Family

4.1 LFM3-VL-2B: Video, Thinking, and a 164M In-House Vision Encoder

LFM3-VL-2B (1.9B parameters) is the vision member, and its three headline capabilities are new for the LFM line: video input (a first for LFM vision models), thinking (step-by-step reasoning before answering, aimed at chart and multi-step visual problems), and spatial/embodied reasoning (localizing objects and reasoning about their relationships — the robotics groundwork). Across 62 vision benchmarks, Liquid reports LFM3-VL-2B averages 71.4, at least 7.4 points ahead of every same-size competitor, essentially matching Gemma 4 12B (71.7) — a model more than six times larger — while surpassing Gemini 3.5 Flash Lite (70.0) and sitting within two points of GPT-5.6 Luna (73.4)^[1].

Each category is the unweighted mean of its benchmark scores; the Average is the unweighted mean across all 62 evaluations. Source: Liquid AI LFM3 launch post.

The quietly important part is the encoder. LFM3-VL ships Liquid's new in-house vision encoder — a 164M-parameter hybrid of convolution and attention that replaces the SigLIP2 encoder used in previous LFM VL models. It uses 3× less compute than SigLIP2, and matches Qwen3.5-2B's vision encoder at 55.9% average accuracy across 13 VLM benchmarks while requiring 2.2× fewer FLOPs. For video, it jointly embeds each pair of sampled frames into one temporal slice, halving the temporal sequence length before further encoding. On-device, Liquid reports the highest prefill throughput and lowest memory footprint in its size class; on a single NVIDIA B300 in BF16, LFM3-VL-2B produces over 12,000 tokens per second^[1].

4.2 LFM3-Audio-2B: LiquidCodec and 50 ms Voice Turnaround

LFM3-Audio-2B takes text or audio in and responds with text, speech, or both, across 18 languages, with audio understanding, general audio reasoning, and spoken tool calling. The reported numbers: English ASR WER of 6.2 (multilingual average 6.0 across nine languages), TTS at UTMOSv2 3.8 with 1.4% WER, VoiceBench 73.1 (vs 55.2 for LFM2.5-Audio-1.5B and 72.9 for Gemma 4 E4B), MMAR audio understanding 50.6, and an internal tool-calling eval of 58.5^[1].

Two new in-house components matter for deployment. The audio encoder is built from sliding-window attention and short-convolution layers (the tri-hybrid recipe again), is fully causal, and can prefill while you speak — time to first token as low as 50 ms after you finish. The LiquidCodec tokenizer compresses each 80 ms of speech into 12 discrete tokens, allows parallel rather than strictly sequential token prediction, and detokenizes at about 87× real time on a single CPU thread (Apple M4 Max). Speech generation faster than real time on one CPU core is what makes a 2B voice assistant realistic on a laptop with no GPU^[1].

4.3 LFM3-Omni-2B: The First Liquid Omni Model

LFM3-Omni-2B (2.2B parameters) is Liquid's first omni model: vision, audio, and language in one model that can see, listen, and talk back. The headline use is cross-modal: spoken questions about images or scenes, video with synchronized audio ("what is happening on screen and what is being said"), and multimodal tool calls that combine what the camera sees with what the user says. Against the omni field, it posts Daily-Omni 70.1 (vs 58.8 for Qwen2.5-Omni-3B and 61.3 for the 7B), WorldSense 47.1, and AV-SpeakerBench 46.8 (vs 40.4 and 43.2), while remaining competitive on OmniBench and spoken visual QA. Liquid's launch examples — a tutor that reads handwritten math and answers by voice, an in-car assistant combining camera and driver speech — are exactly the 2.2B-on-device form factor this architecture targets^[1].

4.4 d1-2B and d1-Omni-2B: Typed Decisions on the LFM3 Backbone

The two decision models extend Liquid's d1 line onto the new backbone. d1-2B (built on LFM3-VL-2B) reads text and pictures; d1-Omni-2B (built on LFM3-Omni-2B) also listens. Instead of writing an answer, they decide — given a situation (a ticket, an invoice, an app screen, a photo, a voice note) and a list of questions, they answer all of them in a single forward pass, each with a calibrated probability^[1].

The big change since d1-3B is typed answers. Beyond yes/no, choices, and scores, both models now support: Point (draw a bounding box around what you describe, or find every instance with a count), Read (pull text, numbers, and dates with the exact source span, saying "not found" when absent), Estimate (a number with a likely range), and Pick (select every applicable option, each with its own probability). Ask as many questions as you like — they're read in one pass over the same input.

d1-Omni-2B keeps nearly all of its accuracy when questions arrive by voice instead of text (DecisionBench 78.2 spoken vs 83.4 written; JevBench 89.0 spoken vs 87.6 written), recognizes everyday sounds at 84.5% on ESC-50, and identifies which of 43 languages is being spoken at 92.5% on FLEURS. For the routing/triage workloads we covered in our decision-model deep dive and the decision-models-vs-rerankers analysis, these are the first LFM-based options that ship with both vision and audio in^[1].

---

5. Training Pipeline

5.1 Pre-training: 22T Tokens in Two Stages

LFM3-1.7B-Base was pre-trained on 22T tokens in two stages. The main stage's data mix, relative to LFM2, adds more scientific papers, repository-level code, considerably more math, and a wider range of knowledge sources — the classic recipe for trading encyclopedic recall for reasoning density, which matches Liquid's own advice to pair the model with retrieval for knowledge work. The final 3.5T tokens are a mid-training stage on curated, higher-quality data with a larger share of agentic traces and diverse tool calls, and this stage also extends the context window to 256K tokens. Mid-training as a distinct phase — rather than a late anneal — is where the 256K window and the agentic priors get installed^[2].

5.2 Post-training: Four Stages to an Agent

1. Supervised fine-tuning (SFT): 1.75T tokens across chat, reasoning, tool calling, and agentic traces — roughly 8% of the pre-training budget, which is enormous by post-training standards and consistent with how much of the benchmark suite is agentic.
2. Mixed reinforcement learning: many domains trained at once — math, code, SQL, structured outputs — with a length penalty in the reward to keep answers concise. The concise-output bias shows up in the MacPaw token-efficiency numbers.
3. Multi-domain on-policy distillation (MOPD): six specialist RL teachers — code, math, tool use, long context, agentic work, and enterprise workflows — distilled back into the single generalist. This is the same "cultivate experts, then merge them" pattern DeepSeek popularized with V4's domain-expert pipeline, adapted to on-policy distillation.
4. Agentic RL: tasks from web browsing to office work run inside real agent harnesses, so the model's deployment environment (multi-step, tool-calling, error-recovering) is also its training environment^[1][2].

Two lineage notes sharpen the picture. First, the multi-teacher distillation is an evolution of a recipe Liquid has published in full before: the LFM2 Technical Report defines a tempered, decoupled Top-K knowledge distillation objective, designed to avoid the support mismatch between teacher and student distributions that breaks naive distillation at small scales. MOPD extends that idea to six on-policy RL teachers, each specialized by reinforcement learning before being distilled back into the single generalist^[13]. Second, LFM3 itself has no technical report yet: the training details above come from the model cards, and the architecture details in Section 2 are quoted from the released config.json and the Apache-2.0 reference implementation. An LFM3 paper with ablations — kernel-size sweeps, head-gate ablations, search-space comparisons — has not been published as of October 29, 2026^[11][12].

The pipeline's shape is the message: LFM3 is post-trained for being an agent first and a chatbot second.

---

6. Inference & Local Deployment

6.1 Edge Performance: Fastest on All Three Measured Devices

Liquid's edge measurements use Q4_0 GGUF files in llama.cpp with 32,768 tokens of context, batch size 1, 100 output tokens, and GPU offload, on three devices: an AMD Ryzen AI Max+ 395 (Radeon 8060S iGPU, Vulkan), an Apple M5 Max (Metal), and an iPhone 17 Pro (A19 Pro, iOS llama.cpp build). The result: LFM3-1.7B prefills at least 2.5× faster and decodes at least 1.8× faster than every compared model on all three devices, while using the least memory. (The comparisons used llama.cpp release b11744; LFM3-1.7B needs a llama.cpp build with LFM3 support.)^[2]

Figure 3 — KV cache memory at a full 128K-token context: 1.6 GB for LFM3-1.7B vs 4.3 GB (Qwen3.5-4B) and 5.6 GB (MiniCPM5-2B). Source: Liquid AI LFM3 announcement (October 2026).

Long context stays affordable because of the conv/SWA-heavy stack: a full 128K-token context needs 1.6 GB of KV cache, against 4.3 GB for Qwen3.5-4B and 5.6 GB for MiniCPM5-2B — 2.7× and 3.5× less respectively. On a phone with ~8 GB of headroom, that difference is the difference between 128K context being theoretical and being usable^[1].

6.2 Checkpoints and the Recommended Stack

Community quantizations of LFM3-1.7B were appearing on Hugging Face within the first day, and the model runs in Docker via Model Runner (docker model run hf.co/LiquidAI/LFM3-1.7B). For an agent box, vLLM or SGLang with the BF16 checkpoint is the straightforward path; for anything battery-powered, the Q4_0 GGUF is the intended target^[2].

6.3 Chat Template, Thinking, and Tool Calling

LFM3 uses ChatML (<|im_start|>…<|im_end|>) with a <think> block: reasoning is enabled by default, and Liquid warns that enable_thinking=False "can be inconsistent, especially for reasoning-heavy tasks." Reasoning from turns before the latest user message is stripped unless you pass preserve_thinking=True^[2].

Tool calls come back as XML, not JSON:

<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>

Schemas go through tools=... on apply_chat_template(); results come back as tool messages, with the assistant's reasoning preserved for the current tool-use turn. The novel bit is tool search: tools marked defer_loading=True stay out of the prompt until your harness's search tool returns their names as tool_reference parts, at which point the template injects their definitions into <additional_tools>. The intended pattern is a catalog of thousands of tools where the model only ever loads the few it finds via search_tools — "tool search to find the right tool among thousands of options," as the launch post puts it. This works with vLLM, SGLang, and LFM3-compatible llama.cpp builds^[1][2].

6.4 Sampling Parameters

Liquid's card specifies two regimes:

# General (chat, tool use, agents, coding)
temperature=0.1, top_k=50, repetition_penalty=1.1

# Math & STEM
temperature=1.0, top_p=0.95, top_k=20

The near-greedy default is worth respecting: small agentic models are more sensitive to sampling noise than frontier ones, and the Section 3 table itself was produced at temperature 0.6 with top_p 0.95 — close to, though not exactly, the near-greedy defaults recommended here.

6.5 Fine-Tuning on a Single GPU

"Made to be specialized" is the launch's organizing claim, and Liquid backs it with TRL notebooks for SFT (extract structured JSON from messy documents), DPO (answer from retrieved documents or decline), GRPO (call the right tools in a 108-tool phone app), and CPT (learn a new language — Turkish — from raw text). The VL demo is the proof point: fine-tuning LFM3-VL-2B on ~3,000 public wind-turbine-blade photos with SFT + GRPO took "a few hours on one A100" and lifted defect classification from under 10% (out of the box, and also for GPT-6 Luna) to 67%, with detection F1 improving from 0.003 to 0.11 (GPT-6 Luna: 0.01). That's the specialized-intelligence pitch in one number: a frontier model loses to a 2B model fine-tuned overnight on one GPU^[1][2].

6.6 Serving Economics: 3.6B Tokens/Day on One B300

On the GPU side, Liquid reports that one NVIDIA B300 running SGLang in BF16 sustains 3.6 billion tokens per day at 1,024 concurrent requests — the highest in the comparison set. Served on Liquid's RIPL runtime, the model holds stable P95 latency under random traffic (the demo use case: online fraud detection). The 300,000-agent Enron demo is the marketing version of the same math: when a model costs ~1 GB and decodes this fast, per-agent token cost stops being the constraint, and agent count becomes the scaling knob^[1].

6.7 The Local-First Reality Check

---

7. Strategic Analysis

7.1 Capability per Parameter Is the New Edge Race

The number Liquid led with — LFM3-1.7B outperforming LFM2.5-2.6B with 35% fewer parameters — is the right number to watch. The edge market doesn't scale by parameter count; it scales by quality per gigabyte and per token-per-second. LFM3's generation gain came from three places: the tri-hybrid layer mix (SWA inserted between convolutions and global attention), the 22T-token run with an agentic mid-training stage, and a four-stage post-training pipeline that ends in real harnesses. Every competitor in this table is now chasing the same ratio, and the MacPaw result (64% task success from a sub-2B model, beating a larger Qwen) suggests the frontier has genuinely moved below 2B^[1][8].

7.2 One Backbone, Many Specialists

The family structure is the strategic bet. Six models from one base mean one architecture to port (llama.cpp, MLX, ONNX, ExecuTorch-style runtimes), one quantization story, one fine-tuning toolchain — and a decision-model line (d1) that inherits backbone improvements for free. The launch partner list reads like a map of where this gets deployed: AMD (Ryzen AI PCs), Qualcomm (Snapdragon), Ambarella (edge vision silicon, via the Cooper Model Garden), Mercedes-Benz (MB.OS in-car assistant), and MacPaw (on-device Mac agents). That's phones, PCs, cameras, cars, and laptops — the "specialized intelligence" distribution channel in five logos^[1][10].

7.3 What LFM3 Means for Local AI

For the local-AI stack specifically, three things change. First, the agent tier has a default: a ~1 GB model that's been explicitly post-trained for OpenClaw/Hermes/Pi-style harnesses is the natural pick for background agents, and it makes "one agent per app" economics plausible on consumer hardware — the workflow our agent-building guide describes, at a tenth of the resource budget. Second, long context on the edge is no longer fantasy: 1.6 GB of KV cache at 128K means a phone can hold a real working session, not a demo. Third, the fine-tune gap widens: when a few hours on one A100 turns a 2B model into a domain expert that beats GPT-6 Luna on a niche task, the "why not just use the API?" answer gets weaker for any high-volume, narrow, privacy-sensitive workload. The counterweight is the knowledge ceiling — GPQA 63.37 and HLE 7.17 say this model still needs retrieval in front of it for anything encyclopedic. The local stack of 2026 shapes up as: a small LFM3-class agent runtime, a retrieval layer, and an API called only when the small model says it's out of its depth.

---

8. What's Still Unknown

1. Serving price — no per-token API pricing for LFM3 was published at launch (the Playground and Liquid API are live; a price page isn't).
2. Per-device edge tables — Liquid's edge speedup claims are directionally clear ("at least 2.5× prefill / 1.8× decode on all three devices"), but the full per-device tok/s tables live in the model cards and haven't been independently replicated yet.
3. Upstream llama.cpp support timeline — LFM3 needs an LFM3-compatible build rather than release b11744; when the support lands upstream (and in Ollama's library) determines friction for the average user.
4. Whether LFM3 scales up — the release is 2B-class only. LFM2 went 350M → 24B-A2B over its life; the LFM3 roadmap beyond 2B (and the relationship to the separately released d1-ultra decision flagship) is unannounced^[5].
5. Thinking-mode off-switch reliability — Liquid itself flags that disabling thinking traces "can be inconsistent, especially for reasoning-heavy tasks."^[2]
6. RULER 128K gap — whether the long-context retrieval deficit vs Qwen3.5 (65.56 vs 84.63/87.11) is a data issue, a training issue, or an architectural tradeoff is unexplained. The config adds a concrete candidate for the architectural reading: the six global layers rotate only 64 of 128 head dims (zero-tail partial RoPE) and the five SWA layers see just 256 tokens, so positional precision for exact needle retrieval is structurally scarcer than in a full-attention model — our hypothesis, not a Liquid explanation (Section 2.4).
7. Training cost — 22T tokens on unspecified hardware; no FLOPs or GPU-hour disclosure.
8. No LFM3 technical report or ablations — unlike LFM2 (arXiv:2511.23404), LFM3 has no paper. The kernel-size choice, the head-gate ablation, the zero-tail RoPE design, and the search-space winners are all unexplained at the ablation level; config.json and the reference implementation are the deepest public sources^[11][12].
9. Head-gate behavior in practice — the per-head sigmoid gates are a central novelty, but no measurements exist of how sparse the gates run, whether heads specialize or switch off permanently, or how gate activity changes after fine-tuning; nothing in the released artifacts answers this.
10. Independent agentic verification — LiveClaw/Claw-Eval margins are the launch's biggest claim and the least third-party-verified; MacPaw's 65-task macOS test is the only external signal so far^[8].

---

9. Bottom Line

---

References