Open-Weight LLM Benchmark Leaderboard November 2026: Qwen, DeepSeek, Llama, Mistral, GLM, Kimi and gpt-oss Compared
The same open-weight model can score 84.1 or 90.6 on Terminal-Bench 2.1, depending only on which agent harness runs it. That spread comes from DeepSeek’s own model card, and it is wider than the gap between most of the models in this open-weight LLM benchmark round-up. In November 2026 the open frontier is crowded: Kimi K3.1 Pro, DeepSeek V4.1 Flash, GLM-5.3, Qwen3.8, Mistral Large 4 and Meta’s Muse Spark Open all claim top scores, while Llama 4 and gpt-oss anchor the smaller end. A ranking that ignores harness, checkpoint date and hardware footprint is not a ranking. This leaderboard reports only published numbers, each with its source. It shows disagreements as ranges and adds what most leaderboards leave out: bytes on disk, GPUs needed, KV-cache cost and license terms.
What this covers: methodology, verified specs for 18 checkpoints, score tables by size class, cross-vendor score disagreements, worked VRAM and KV-cache math, a license comparison and a use-case decision guide.
Context and Background
Five months ago we described the June 2026 flood of open-weight releases. The flood did not slow down. Since then Moonshot shipped Kimi K3 and then Kimi K3.1 Pro (Hugging Face repo created 2 November 2026). DeepSeek released V4-Pro-0813, V4.1-Flash on 10 September and a 1017 refresh. Z.ai followed GLM-5.2 with GLM-5.3 and GLM-5.3-Flash in late August. Alibaba’s Qwen team released Qwen3.8 at three sizes. Mistral released Mistral Large 4 on 30 September. Meta released Muse Spark Open on 15 October.
Two familiar names have gone quiet. On Hugging Face, the newest Llama language models under the meta-llama organisation are still Llama 4 Scout and Maverick, released on 5 April 2025. Meta’s newer open release lives under a different organisation (meta-models) and a different brand. OpenAI’s only open-weight LLMs are still gpt-oss-120b and gpt-oss-20b from August 2025, plus the gpt-oss-safeguard fine-tunes. The title still says “Llama” and “gpt-oss” because readers keep searching for those names. In this leaderboard they work as baselines, not contenders.
The open-weight LLM benchmark landscape has moved as well. The scores the community used to compare on, such as MMLU-Pro, SWE-bench Verified and AIME, have mostly disappeared from frontier model cards. Their place has gone to agentic suites: Terminal-Bench 2.1 to 4.0, DeepSWE v1.1, SWE-bench Pro, Toolathlon, and Artificial Analysis’s composite index, now at version 5.0.1 on Artificial Analysis’s open-weights page. Agentic suites measure something closer to real work. They are also far more sensitive to the scaffolding around the model, which matters for everything that follows. We covered the single-model view in our GLM-5.2 open-weight benchmark analysis. This post is the cross-family open-weight LLM benchmark.
How to Read an Open-Weight LLM Benchmark in November 2026: Score, Harness, Footprint
A useful open-weight LLM benchmark comparison in late 2026 has three parts: a score with its harness and checkpoint date, a hardware footprint (weight bytes plus KV cache per token), and the license terms for your deployment model. A single number without those three is marketing. Read vendor tables as ranges, not ranks.

Figure 1: How this leaderboard turns vendor claims into a shortlist, with harness normalisation, footprint math and license filtering.
Figure 1 shows the pipeline used here. There are three score inputs: each vendor’s own model card, the comparison columns other vendors print about their rivals, and the Artificial Analysis index. Each score is tagged with the harness, the reasoning-effort setting and the checkpoint it applies to. Scores that share a harness and version count as comparable. Scores that do not are shown as a range. Every model is then placed in a size class, its memory footprint is computed from Hugging Face repo metadata, and its license is checked against how you plan to deploy it.
Every number here is a published number
All open-weight LLM benchmark data here comes from primary sources fetched on 16 November 2026. Those sources are the Hugging Face model cards and LICENSE files for each checkpoint, the gpt-oss model card on arXiv, the Llama 4 Maverick card, and the public Artificial Analysis open-weights page. Parameter counts come from the model cards and from the Hugging Face API’s safetensors metadata. Disk sizes were summed from the repository file listings. Where a card does not publish a metric, the table says “not reported”. We did not run our own evaluations and we do not interpolate. Third-party aggregator sites were used only to find which models to check, never as a source for a number.
Vendor tables disagree, so ranges beat ranks
Every 2026 model card has its own open-weight LLM benchmark table, and the tables disagree about the same models. Moonshot’s Kimi K3 card reports 88.3 on Terminal-Bench 2.1. Meta’s Muse Spark Open card scores the same Kimi K3 at 85.0 using the Terminus 2 agent. Z.ai’s own GLM-5.3 card does not report GPQA Diamond at all. DeepSeek’s V4.1 card lists GLM-5.3 at 88.1, while Meta’s and Moonshot’s cards both list it at 91.7. That is a 3.6-point spread on a multiple-choice science benchmark, for one checkpoint.
The most dramatic example compares two dates. GLM-5.2’s June card listed DeepSeek-V4-Pro at 8 on DeepSWE and 64 on Terminal-Bench 2.1 with Terminus-2. GLM-5.3’s card later listed DeepSeek-V4-Pro-0813 at 62.7 and 87.9 on the same two benchmarks. Some of that gap comes from a newer checkpoint, and some from different harnesses. Neither card can tell you how much is which. The rule this leaderboard follows is simple: if two published sources disagree by more than a point, show both.
Composite indices drift between versions
Composite open-weight LLM benchmark indices drift as well. The Artificial Analysis open-weights page, under Intelligence Index v5.0.1, currently shows Kimi K3.1 Pro at 46, DeepSeek V4.1 Flash 1017 (Max) at 33, GLM-5.3 (Max) at 32, Mistral Large 4 (High) at 26 and Qwen3.8 27B (Xhigh) at 20. Meta’s Muse Spark Open card prints a column labelled “Artificial Analysis Intelligence Index” with GLM-5.3 at 45, Kimi K3 at 44 and Mistral Large 4 at 38. Mistral’s own card says Large 4 scores 26 on v5. Our inference is that Meta’s column uses an earlier index version, but the card does not say so, so treat that as unverified. Either way, an index score without its version number is not portable from one table to another.
Footprint belongs on the leaderboard
The third part is the one most leaderboards skip. A 2.8-trillion-parameter model and a 27-billion-parameter model may be a few points apart on a given suite, yet they cost very different amounts to run. One needs two server nodes and the other fits on a single GPU. For anyone running models themselves, “points per GPU” decides more than “points”. Every table below therefore puts the measured checkpoint size on disk next to the scores. The VRAM section then turns those sizes into hardware.
Results: Specs and Scores by Size Class
The 18 checkpoints in this open-weight LLM benchmark fall into three size classes by total parameters. Table 1 lists the verified specs. “Active” means parameters used per token in a Mixture-of-Experts (MoE) model. “Disk” is the sum of the top-level .safetensors files in the official repository on 16 November 2026, which reflects the precision each vendor actually ships.
Table 1: Verified specifications (sources: official Hugging Face model cards and repo metadata)
| Model | Developer | Total / active params | Context | Shipped format | Disk (GB) | License |
|---|---|---|---|---|---|---|
| Kimi K3.1 Pro | Moonshot AI | 2.8T / 104B | 1,048,576 | MXFP4 weights (QAT) | 1,561 | Kimi K3.1 Pro License |
| Kimi K3 | Moonshot AI | 2.8T / 104B | 1,048,576 | MXFP4 weights (QAT) | 1,561 | Kimi K3 License |
| Qwen3.8-2.4T-A95B | Alibaba Qwen | 2.4T / 95B | 262,144 native, 1,010,000 extended | BF16; FP8 repo | 4,892 BF16; 2,496 FP8 | Qwen3.8-Max License |
| DeepSeek-V4-Pro-0813 | DeepSeek | 1.6T / 49B | 1M | FP8 with FP4 experts | 893 | MIT |
| Mistral Large 4 | Mistral AI | ~1T / ~52B incl. embeddings | 1M | FP8; NVFP4 repo | 1,061 FP8; 610 NVFP4 | Mistral Ecosystem Open-Weight |
| Muse Spark Open | Meta | 816B / 42B | 1M | BF16; NVFP4 repo | 533 NVFP4 | Muse Model License |
| GLM-5.3 | Z.ai | 753B (HF count) / not stated | 1M (per GLM-5.2 card) | FP8; BF16 repo | 756 FP8; 1,507 BF16 | GLM-5.3 License |
| DeepSeek-V4.1-Flash | DeepSeek | 552B backbone / 8B prefill, 16B decode | 1M | FP8 with FP4 experts | 510 | MIT |
| GLM-5.3-Flash | Z.ai | 320B / 18B | not stated on card | FP8; BF16 repo | 328 FP8 | MIT |
| DeepSeek-V4-Flash-0731 | DeepSeek | 284B / 13B | 1M | FP8 with FP4 experts | 167 | MIT |
| Qwen3.8-Flash-Next | Alibaba Qwen | 125B / 6B, plus 51B n-gram embedding, 4B MTP | 262,144 native, 1M extended | BF16; FP8 repo | 360 BF16 | Qwen Community License 1.0 |
| gpt-oss-120b | OpenAI | 117B / 5.1B | 131,072 | MXFP4 MoE weights | 65 | Apache 2.0 |
| Qwen3.8-27B | Alibaba Qwen | 27B dense | 262,144 native, 1M extended | BF16; FP8 repo | 56 BF16; 31 FP8 | Apache 2.0 |
| gpt-oss-20b | OpenAI | 21B / 3.6B | 131,072 | MXFP4 MoE weights | 14 | Apache 2.0 |
| Llama 4 Maverick | Meta | 400B / 17B | 1M | BF16; FP8 repo | not measured | Llama 4 Community |
| Llama 4 Scout | Meta | 109B / 17B | 10M | BF16 | not measured | Llama 4 Community |
GLM-5.3 shares its base model with GLM-5.2, and its card says all of the gains come from post-training. The 1M context figure comes from the GLM-5.2 card. The DeepSeek V4.1 card describes a separate 196B-parameter “Engram” conditional memory and does not say whether the 552B backbone figure includes it, so we do not compute bytes per parameter for that model.
Frontier class: more than 1 trillion parameters
Table 2: Frontier-class open-weight LLM benchmark scores (max or highest reasoning effort; source in brackets)
| Model | GPQA Diamond | Terminal-Bench 2.1 | DeepSWE v1.1 | SWE-bench Pro | AA Index v5 |
|---|---|---|---|---|---|
| Kimi K3.1 Pro | 94.2 [Moonshot] | not reported (TB 4.0: 65.7) | 75.5 [Moonshot] | not reported | 46 [AA] |
| Kimi K3 | 93.5 [Moonshot] | 88.3 [Moonshot]; 85.0 Terminus 2 [Meta] | 67.5 [Moonshot] | not reported | not listed on AA page |
| Qwen3.8-2.4T-A95B (“Max”) | 92.6 [Qwen]; 93.5 [Meta] | 86.6 [Qwen]; 82.0 Terminus 2 [Meta] | 56.6 [Qwen] | 67.7 [Qwen] | not listed on AA page |
| DeepSeek-V4-Pro(-0813) | 92.4 [DeepSeek] | 87.9 [DeepSeek, Z.ai] | 62.7 [DeepSeek] | not reported | not listed on AA page |
| Mistral Large 4 | not reported on card | not reported on card | 62, preferred harness [Meta] | not reported | 26 [Mistral, AA] |
Kimi K3.1 Pro is the clearest leader in this class. It is first on the Artificial Analysis v5 index among open-weight models, and Moonshot’s card puts it ahead of DeepSeek-V4.1-Flash-1017 (42.3) and GLM-5.3 (41.8) on Terminal-Bench 4.0 at 65.7. Mistral Large 4 trails on composite scores. Mistral says it scored 48% on the Artificial Analysis Coding Agent Index, “effectively tied” with GLM-5.3, while using 47% fewer tokens per task. Token efficiency is a cost lever that never shows up in accuracy tables.
Large class: 250 billion to 1 trillion parameters
Table 3: Large-class open-weight LLM benchmark scores
| Model | GPQA Diamond | Terminal-Bench 2.1 | DeepSWE v1.1 | AA Index v5 |
|---|---|---|---|---|
| GLM-5.3 | 88.1 [DeepSeek]; 91.7 [Meta, Moonshot] | 88.2 Claude Code [Z.ai]; 83.9 Terminus 2 [Meta] | 66.9 [Z.ai] | 32 [AA] |
| DeepSeek-V4.1-Flash | 90.9 [DeepSeek]; 92.2 for 1017 [Moonshot] | 90.6 DSH Minimal [DeepSeek]; 84.1 to 90.6 across harnesses | 74.2 [DeepSeek]; 77.0 for 1017 [Moonshot] | 33 for 1017 [AA] |
| Muse Spark Open | 89.0 [Meta] | 76.4 Terminus 2 [Meta] | 60, preferred harness [Meta] | not listed on AA page |
| DeepSeek-V4-Flash | 89.9 [DeepSeek] | 82.7 [DeepSeek] | 54.4 [DeepSeek] | not listed on AA page |
| GLM-5.3-Flash | chart image only | chart image only | chart image only | not listed on AA page |
DeepSeek-V4.1-Flash is the standout in this class. DeepSeek’s table puts it above GLM-5.3 and Kimi K3 on Terminal-Bench 2.1, DeepSWE and AutomationBench. It does that with 8B active parameters during prefill and 16B during decode. Muse Spark Open scores lower on coding but leads its own table on safety: 0.5 on StrongREJECT v2 (lower is better), against 62.9 to 86.4 for the other models in Meta’s table. GLM-5.3-Flash publishes its scores only as an image. Z.ai says in text that it beats GLM-5.2 “at one-tenth the price”. We have not transcribed numbers from the chart.
Compact class: under 200 billion parameters
Table 4: Compact-class open-weight LLM benchmark scores
| Model | GPQA Diamond | Coding | Other published | Source |
|---|---|---|---|---|
| Qwen3.8-Flash-Next | 91.7 | SWE-bench Pro 62.5; DeepSWE 58.7; LiveCodeBench v6 91.9 | Toolathlon 73.5 | Qwen card |
| Qwen3.8-27B | 89.2 | SWE-bench Pro 61.7; TB 2.1 (Terminus) 73.0; LiveCodeBench v6 90.3 | AA v5: 20 | Qwen card; AA |
| gpt-oss-120b (high) | 80.1 no tools | SWE-bench Verified 62.4 | AIME 2025 with tools 97.9; MMLU 90.0 | arXiv 2508.10925 |
| gpt-oss-20b (high) | 71.5 no tools | SWE-bench Verified 60.7 | AIME 2025 with tools 98.7; MMLU 85.3 | arXiv 2508.10925 |
| Llama 4 Maverick | 69.8 | LiveCodeBench 43.4 | MMLU-Pro 80.5 | Meta card |
| Llama 4 Scout | 57.2 | LiveCodeBench 32.8 | MMLU-Pro 74.3 | Meta card |
This class has moved the most since the last open-weight LLM benchmark round. In its own table, Qwen3.8-27B, a dense model that fits on one 80 GB GPU in BF16, scores 61.7 on SWE-bench Pro. Claude Opus 4.6 Max scores 53.4 in the same table. Qwen3.8-Flash-Next activates 6B parameters per token and scores 91.7 on GPQA Diamond. Llama 4’s numbers are 19 months old, and its LiveCodeBench window (October 2024 to February 2025) is not the v6 window Qwen reports. Treat it as a historical baseline.
Methodology caveats
Four caveats apply to every open-weight LLM benchmark table above. First, nearly all scores are vendor-reported, measured at “max” or “xhigh” reasoning effort with temperature 1.0. Production deployments often run at lower effort. Second, MMLU-Pro appears only for Llama 4 and for DeepSeek’s base models (V4.1-Flash-Base 74.1, V4-Pro-Base 73.5, V4-Flash-Base 68.3). Base-model scores are not comparable with instruct-model scores. Third, the gpt-oss and Llama 4 numbers use older benchmark versions than the 2026 cards. Fourth, benchmark contamination is not audited by any of these sources. A 1-point gap on any line is noise.
Worked VRAM and Serving Math for a Self-Hosted LLM Comparison
An open-weight LLM benchmark score tells you which model is smarter. Footprint tells you which one you can afford to run. Use this section as a capacity-planning tool. All hardware sizing here is illustrative, built on nominal per-GPU memory figures: 80 GB for an H100, 141 GB for an H200, and about 180 GB per B200 in an 8-GPU DGX-class node. Engine overheads vary by version and settings, so check them on your own stack.

Figure 2: The memory budget for serving an open-weight model. Weights are fixed, KV cache grows with context and concurrency.
Figure 2 splits the budget into two parts. Weight memory is fixed: parameters times bytes per parameter in the checkpoint format. KV-cache memory varies: bytes per token, times context length, times the number of concurrent sequences. Add a runtime overhead for CUDA graphs, activations and fragmentation. Then ask whether the total fits on one 8-GPU node, which avoids slow inter-node links for tensor or expert parallelism.
Step 1: weight bytes per parameter
Dividing the measured disk size by the parameter count gives effective bytes per parameter. This number already includes scale factors and any layers the vendor kept at higher precision:
- Kimi K3: 1,560.9 GB ÷ 2.8T ≈ 0.56 bytes per parameter, or about 4.5 bits. MXFP4 stores 4-bit values with a shared 8-bit scale per 32-element block, about 4.25 bits. The remainder comes from attention, embeddings and the 401M-parameter vision encoder at higher precision.
- DeepSeek-V4-Pro-0813: 892.7 GB ÷ 1.65T ≈ 0.54 bytes per parameter, consistent with FP4 experts and FP8 elsewhere.
- GLM-5.3: 755.6 GB ÷ 753B ≈ 1.0 byte per parameter, which is FP8 throughout. The BF16 repo (1,506.7 GB) is double that, as expected.
- Mistral Large 4: about 1.06 bytes per parameter in FP8, falling to about 0.61 in the NVFP4 repo. Mistral says the NVFP4 version gives the “same performance” on Blackwell GPUs.
- Muse Spark Open NVFP4: 533 GB ÷ 816B ≈ 0.65 bytes per parameter. Meta’s card nevertheless says the model needs “at least 700GB of VRAM for NVFP4 and 2TB for BF16”. That gap is the runtime and KV headroom Meta budgets for.
Step 2: node fit
Comparing those sizes with 8-GPU nodes (640 GB of H100 memory, 1,128 GB of H200, about 1,440 GB of B200) gives the following illustrative fits:
| Model | Weights | 8×H100 (640 GB) | 8×H200 (1,128 GB) | 8×B200 (~1,440 GB) |
|---|---|---|---|---|
| Kimi K3 / K3.1 Pro | 1,561 GB | no | no | no, needs multi-node or larger-memory GPUs |
| Qwen3.8-2.4T FP8 | 2,496 GB | no | no | no, multi-node |
| Mistral Large 4 FP8 | 1,061 GB | no | only ~67 GB left | yes, ~380 GB left |
| Mistral Large 4 NVFP4 | 610 GB | not applicable (Blackwell format) | not applicable | yes, ~830 GB left |
| DeepSeek-V4-Pro-0813 | 893 GB | no | yes, ~235 GB left | yes |
| GLM-5.3 FP8 | 756 GB | no | yes, ~372 GB left | yes |
| Muse Spark Open NVFP4 | 533 GB | not applicable | not applicable | yes; Meta says ≥700 GB |
| DeepSeek-V4.1-Flash | 510 GB | tight, ~130 GB left | yes, ~618 GB left | yes |
| Qwen3.8-Flash-Next BF16 | 360 GB | yes, ~280 GB left | yes | yes |
| GLM-5.3-Flash FP8 | 328 GB | yes | yes, or 4×H200 | yes |
| gpt-oss-120b | 65 GB | one GPU (per OpenAI) | one GPU | one GPU |
| Qwen3.8-27B BF16 / FP8 | 56 / 31 GB | one GPU | one GPU | one GPU |
| gpt-oss-20b | 14 GB | 16 GB device class (per OpenAI) | n/a | n/a |
The trillion-parameter Kimi models are the only leaders that cannot be served from a single standard 8-GPU node in their shipped format. That is the real price of their lead on the AA index.
Step 3: KV cache per token, the hidden scaling term
The KV cache is the per-token attention state a server keeps for every active sequence. For standard grouped-query attention, the bytes per token are 2 (key and value) × full-attention layers × KV heads × head dimension × bytes per element. The 2026 architectures cut this term aggressively, and it decides concurrency.
Qwen3.8-27B (from its config): only 16 of 64 layers use full attention (one in four). The rest are Gated DeltaNet linear-attention layers, which keep a fixed-size state. That gives 2 × 16 × 4 × 256 × 2 bytes = 65,536 bytes per token in BF16. At the native 262,144-token context, one sequence holds about 17.2 GB of KV cache, or about 8.6 GB with an FP8 KV cache. The fixed DeltaNet state comes on top and is not computed here.
gpt-oss-120b (from its config): 36 layers alternate between sliding-window attention (128-token window) and full attention, with 8 KV heads of dimension 64. The 18 full layers cost 2 × 18 × 8 × 64 × 2 = 36,864 bytes per token. At 131,072 tokens that is about 4.8 GB per sequence. The sliding layers add under 5 MB.
DeepSeek-V4.1-Flash (from its card): with compressed sparse attention, cross-layer KV sharing and an FP4 main KV cache, DeepSeek reports a global KV cache of 890 bytes per token, about one-quarter of V4-Flash. A full 1M-token sequence needs about 0.89 GB, plus the bounded sliding-window replay state.
For contrast, take a hypothetical, illustrative dense model with full attention on all 64 layers, 8 KV heads of dimension 128, in BF16. It needs 262,144 bytes per token, which is 262 GB for a single 1M-token sequence. This is why “1M context” on a spec sheet means little until you know the KV cost per token. Our deep-dives on multi-head latent attention versus GQA and prefix caching with RadixAttention and vLLM cover how serving engines exploit this further.
Step 4: a worked concurrency estimate (illustrative)
Put DeepSeek-V4.1-Flash on 8×H100. That leaves about 130 GB after weights. Assume 25% goes to runtime overhead (an assumption, not a measurement), which leaves about 97 GB for KV. At 890 bytes per token that is roughly 109 million tokens of global KV, enough on paper for about a hundred concurrent 1M-token agent sessions. In practice, sparse-indexer caches, replay windows and scheduler limits will bind first. The point is the order of magnitude. A GQA model of similar quality would fit one or two such sessions on the same hardware.
Decode speed follows active parameters. Each generated token streams the active weights through memory. The Artificial Analysis page measured Kimi K3.1 Pro (104B active) at 30 output tokens per second, GLM-5.3 at 88, and DeepSeek V4.1 Flash 1017 (16B active in decode) at 174, on whatever hardware each provider uses. Provider hardware differs, so these numbers are directional only. The ordering still matches the physics.

Figure 3: One checkpoint, six harnesses, a 6.5-point spread. Data from DeepSeek’s V4.1-Flash card, N=3 samples per task, max effort.
Figure 3 shows why the harness belongs in every row of an open-weight LLM benchmark table. DeepSeek ran the same V4.1-Flash weights through Claude Code, Codex, OpenCode, Pi, mini-SWE and its own DSH scaffolds. Terminal-Bench 2.1 ranged from 84.1 to 90.6. DeepSWE ranged from 65.5 to 74.2. That spread is bigger than the gap between the third-ranked and eighth-ranked models in most of the tables above.
License Comparison: What “Open Weight” Allows in November 2026
The licenses have split into three families, and the split matters more than any score for anyone who runs a model as an API. “Model as a Service” (MaaS) is the key term. Most of the custom licenses define it as giving a third party API access with “meaningful control over the inputs, parameters, or training data”. Most of them exclude end-user products where the model is only embedded in a feature.
| License | Models | Commercial MaaS condition | Attribution |
|---|---|---|---|
| MIT | DeepSeek V4, V4.1; GLM-5.2; GLM-5.3-Flash | none | keep the notice |
| Apache 2.0 | Qwen3.8-27B; gpt-oss-120b, 20b | none | notice plus patent terms |
| GLM-5.3 License | GLM-5.3 | above US$10 billion revenue in 12 months, pass Z.AI security review | notice |
| Kimi K3 / K3.1 Pro License | Kimi K3, K3.1 Pro | above US$20 million revenue in 12 months, separate agreement | display “Kimi K3” above 100M MAU or US$20M monthly revenue |
| Muse Model License | Muse Spark Open | above US$20 million revenue in 12 months, notify Meta and sign agreement | “Built with Muse Spark Open” always; usage policy applies |
| Qwen3.8-Max License | Qwen3.8-2.4T-A95B | MaaS or “AI Work Assistant” above US$50 million revenue, separate license | model name above 100M MAU or US$20M monthly revenue |
| Qwen Community License 1.0 | Qwen3.8-Flash-Next | any MaaS or AI Work Assistant business, separate license | as above |
| Mistral Ecosystem Open-Weight | Mistral Large 4 | any MaaS business, commercial license | notice |
| Llama 4 Community | Llama 4 Scout, Maverick | above 700M MAU on release date, license from Meta | “Built with Llama” |
All of these exemptions cover internal use, defined as use that does not expose the model or its outputs to third parties. Qwen’s “AI Work Assistant” clause is new and specific. It covers independent coding or office-productivity assistants, which is exactly the market many startups are building for. For the general framework, see our explainer on AI model licenses and what commercial use really allows. None of this is legal advice. Read the LICENSE file in the exact repository you download.
Trade-offs, Gotchas, and What Goes Wrong
Max-effort scores do not describe your production setting. Every frontier card reports “max” or “xhigh” reasoning with very large output budgets: up to 163,840 generated tokens for HLE in Z.ai’s footnotes, and at least 256K max tokens recommended by DeepSeek. At low effort the rankings can reorder, and nobody publishes those tables. Benchmark at the effort level you will actually pay for.
Chat templates are part of the model. DeepSeek V4.1 ships no Jinja chat template, only a Python reference encoder and a Rust toolkit. GLM-5.3 defaults clear_thinking to false, which keeps old reasoning in context, so chat applications must pass clear_thinking=true. gpt-oss works only with the harmony format. Get these wrong and an open-weight LLM benchmark rerun on your own stack will come in below the card, and the cause will be integration, not the model.
Quantization changes more than speed. Kimi K3 and gpt-oss were trained with quantization-aware training (QAT) at MXFP4, so their published scores are the quantized scores. For models released in BF16 or FP8, any community 4-bit build you download is a different model, and its scores are unknown until you measure them. Meta’s card says plainly that “quantized inference may show minor quality differences in edge cases”. Our comparison of GGUF, AWQ, GPTQ and FP8 explains why.
Hybrid attention complicates serving engines. Qwen3.8 mixes Gated DeltaNet with gated or sparse attention. Kimi mixes KDA with gated DSA. DeepSeek V4.1 adds an encoder-decoder split and an “Engram” memory. Prefix caching, chunked prefill and speculative decoding each need explicit support for these layer types. Pin engine versions that the vendor recipe names.
Safety varies widely. Meta reports StrongREJECT v2 adaptive-attack scores of 62.9 to 86.4 for the other open models in its table, against 0.5 for Muse Spark Open. Those are Meta’s numbers about its competitors, and they need independent confirmation. Even so, the gap means that guardrails around a self-hosted model are your job.

Figure 4: A license triage flow. Internal use is broadly permitted; API resale is where licenses diverge.
Practical Recommendations
Start from your deployment shape, not the open-weight LLM benchmark leaderboard. If you need one GPU, the choice is effectively Qwen3.8-27B (Apache 2.0, strongest compact coding scores) or gpt-oss-120b (Apache 2.0, natively MXFP4, 65 GB). If you have one 8-GPU H100 or H200 node and want frontier-level agentic coding under MIT, DeepSeek-V4.1-Flash is the best published balance of score, KV cost and license. If you have the budget for multi-node Blackwell and need the top of the AA index, Kimi K3.1 Pro leads, under a license with a US$20 million MaaS threshold. For a broader view of whether to self-host at all, compare against hosted per-token prices in our LLM API pricing for November 2026 post.
Decision guide by use case:
| Use case | First pick | Alternative | Why |
|---|---|---|---|
| Agentic coding, one node, permissive license | DeepSeek-V4.1-Flash | GLM-5.3 (8×H200) | MIT; DeepSWE 74.2; 890 B/token KV |
| Highest composite score, cost secondary | Kimi K3.1 Pro | Qwen3.8-2.4T | AA v5 46; needs multi-node |
| Single 80 GB GPU | Qwen3.8-27B | gpt-oss-120b | SWE-bench Pro 61.7 vs Verified 62.4, both Apache |
| 16 GB laptop or edge box | gpt-oss-20b | Qwen3.8-27B at 4-bit (measure it) | official 16 GB claim |
| EU languages, vision grounding | Mistral Large 4 | Muse Spark Open | 160+ languages; check MaaS terms |
| Consumer-facing, safety-critical | Muse Spark Open | any model plus external guardrails | lowest published StrongREJECT score |
| API reseller | DeepSeek V4.x or GLM-5.3-Flash | gpt-oss | no MaaS clause |
Before you commit, work through this checklist:
- Rerun your top two candidates as a private open-weight LLM benchmark on your own tasks, in your production harness, at your production reasoning effort.
- Record the checkpoint ID (for example
-0813or-1017), engine version and chat-template settings with every result. - Compute weights plus KV per sequence at your target context before you buy or reserve GPUs.
- Read the LICENSE in the exact repository, including MaaS and attribution thresholds.
- Plan for re-evaluation every quarter, because checkpoints in this field turn over every 6 to 10 weeks.
Frequently Asked Questions
What is the best open-weight LLM in November 2026?
By composite open-weight LLM benchmark score, Kimi K3.1 Pro leads. It is first among open-weight models on the Artificial Analysis Intelligence Index v5.0.1 with 46, ahead of DeepSeek V4.1 Flash 1017 at 33 and GLM-5.3 at 32. “Best” depends on constraints, though. Kimi K3.1 Pro needs about 1.56 TB just for weights. DeepSeek-V4.1-Flash fits on one 8-GPU node, uses an MIT license and leads its own agentic coding tables. For a single GPU, Qwen3.8-27B is the strongest published option.
Qwen vs DeepSeek vs Llama: which should I self-host?
For most new deployments, choose between Qwen and DeepSeek. Llama 4 Scout and Maverick date from April 2025 and score well below 2026 models: Maverick has 69.8 on GPQA Diamond against 89.2 for Qwen3.8-27B. DeepSeek-V4.1-Flash suits multi-GPU servers that need long context, because its KV cost is very low. Qwen3.8-27B suits single-GPU or cost-sensitive setups under Apache 2.0. Meta’s current open model is Muse Spark Open, which is not branded Llama.
Why do benchmark scores for the same model differ between sources?
Because agentic benchmarks measure the model plus its harness, its reasoning effort, its context limit and its checkpoint. DeepSeek’s own card shows one checkpoint scoring 84.1 to 90.6 on Terminal-Bench 2.1 across six scaffolds. Meta scores Kimi K3 three points lower than Moonshot does with a different agent. Composite indices also change between versions. When sources disagree by more than a point, treat the true value as a range and run your own open-weight LLM benchmark.
How much GPU memory do I need to run these models?
Start with checkpoint size: roughly 2 bytes per parameter for BF16, about 1 for FP8, and about 0.55 to 0.65 for FP4 formats once scales are included. Then add KV cache: bytes per token times context times concurrent sequences, plus overhead. In practice gpt-oss-120b fits one 80 GB GPU. GLM-5.3 FP8 and DeepSeek-V4-Pro fit one 8×H200 node. Kimi K3 at about 1.56 TB needs more than one standard 8-GPU node.
Is gpt-oss still worth using?
Yes, for specific jobs. gpt-oss-120b runs on a single 80 GB GPU, uses Apache 2.0, and OpenAI’s model card reports 62.4 on SWE-bench Verified and 97.9 on AIME 2025 with tools at high effort. Its 5.1B active parameters make it fast and cheap, and it remains a useful open-weight LLM benchmark floor. It is now far behind the 2026 leaders on agentic suites, though, and OpenAI has not released a newer open-weight LLM. Treat it as a dependable, permissively licensed baseline, not a frontier contender.
Are these open-weight models truly open source?
Mostly not, in the strict sense. They release weights and inference code, not training data or full training pipelines. Licenses range from MIT and Apache 2.0 (DeepSeek, GLM-5.3-Flash, Qwen3.8-27B, gpt-oss) to custom terms that restrict model-as-a-service businesses above revenue thresholds (Kimi, Muse, Qwen3.8-Max, GLM-5.3), and to licenses that require a commercial agreement for any API resale (Mistral Large 4, Qwen3.8-Flash-Next). “Open weight” is the accurate term.
Further Reading
- GLM-5.2 open-weight LLM benchmark analysis — the single-model deep dive this leaderboard builds on.
- The open-weight model flood of June 2026 — the release wave that set up today’s field.
- LLM prefix caching: RadixAttention vs vLLM automatic prefix caching — how KV reuse changes the serving math above.
- LLM API pricing, November 2026 — the hosted baseline to compare self-hosting against.
- DeepSeek-AI, DeepSeek-V4.1-Flash model card and technical report, Hugging Face, 2026.
- OpenAI, gpt-oss-120b and gpt-oss-20b Model Card, arXiv:2508.10925, 2025.
By Riju — about
