MiniCPM5-2B Reality Check: 46.4 SWE-bench Verified, 14.4 Pro, but AA v4.2 Is 15

What MiniCPM5-2B actually is

OpenBMB released MiniCPM5-2B on September 7, 2026, the second model in the MiniCPM5 series. The release is a dense, text-only causal language model built on the standard LlamaForCausalLM architecture. The official model card gives an exact size of 2,516,756,480 total parameters, including 1,981,982,720 non-embedding parameters, with 42 layers, grouped-query attention using 16 query heads and 2 key/value heads, and a 131,072-token context window.

The practical release is unusually complete for a compact model. OpenBMB publishes the final BF16 RL+OPD checkpoint, SFT-only, mid-training and base checkpoints, plus GGUF, 4-bit MLX, 4-bit GPTQ and a DSpark draft model for speculative decoding. The model and repository are released under Apache 2.0, and OpenBMB also links the UltraData training-data family used across pre-training, SFT and RL.

Those details make the release more reproducible than a weights-only launch, but they do not make every benchmark claim independently reproduced. OpenBMB's model card mixes its own internal reproductions with selected scores imported from Artificial Analysis, and the distinction matters.

SWE-bench Verified and SWE-bench Pro are separate results

OpenBMB reports 46.4 on SWE-bench Verified and 14.4 on SWE-bench Pro for MiniCPM5-2B. Both numbers are listed in the model card's “Coding Agent” section and are not marked with the dagger symbol used for Artificial Analysis results. OpenBMB explicitly says that non-dagger scores are reproduced internally, so these two SWE-bench numbers should be treated as provider-run evaluations, not neutral third-party reproductions.

The gap between 46.4 and 14.4 is not a contradiction. SWE-bench Verified and SWE-bench Pro are different benchmark suites with different task pools and difficulty profiles. A score on one must not be transferred to the other, averaged with it, or described as if both measure the same sample.

The current model card does not expose enough detail to independently reconstruct those two runs from the headline table alone. It does not state, next to the SWE-bench rows, the exact agent scaffold, repository subset for Pro, task count actually attempted, retry policy, per-task token budget, patch-validation procedure or the full raw trajectories. Until those details or a pinned independent rerun are published, the strongest defensible wording is that OpenBMB reports 46.4 Verified and 14.4 Pro.

Terminal-Bench is a different, independently sourced signal

For Terminal-Bench v2.1, MiniCPM5-2B is listed at 8.6. That row carries OpenBMB's dagger marker, and the model card says daggered scores come from the official Artificial Analysis release. Terminal-Bench is therefore a different evidence path from OpenBMB's internally reproduced SWE-bench results.

This distinction also prevents a common ranking error. A Terminal-Bench score cannot be ranked directly against SWE-bench Verified or SWE-bench Pro as if all three were a single percentage scale. They test different agent environments and success criteria.

Artificial Analysis v4.2 gives an independent composite score of 15

Artificial Analysis currently scores MiniCPM5-2B at 15 on its Intelligence Index v4.2, placing it #1 among 47 models in its tiny open-weight comparison class at the time of this verification. Artificial Analysis says the evaluation consumed about 57 million output tokens and lists the model as text-only, reasoning-capable, Apache-2.0 open weights with a roughly 131K context window.

That independent 15 is useful, but it is not comparable to OpenBMB's “53.9 average”. Artificial Analysis v4.2 is a weighted composite of ten evaluations: AA-Briefcase, GDPval-AA v2, τ³-Banking, Terminal-Bench v2.1, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience and AA-LCR v1.1. OpenBMB's 53.9 is an arithmetic-style average across a much broader launch table containing coding, mathematics, instruction following, general knowledge, long-context, tool-use and agent benchmarks whose raw scales and protocols differ.

So “53.9” should not be described as 53.9% intelligence, and the difference between 53.9 and 15 is not evidence that one evaluator found the model four times weaker. They are different composites.

What the rest of OpenBMB's benchmark table shows

Within OpenBMB's comparison, MiniCPM5-2B scores 69.1 on LiveCodeBench v6, 68.0 on LCB-Pro 25Q2 Easy, 17.5 on LCB-Pro 25Q2 Medium, 32.5 on OJBench, 86.5 on AIME 2025, 86.5 on AIME 2026, 94.6 on MATH-500, 66.6 on BFCL v4, 97.1 on τ²-Bench Telecom, 88.7 on GAIA Text-103 and 59.2 on Claw-Gym.

Some launch-table values are sourced from Artificial Analysis instead of OpenBMB reruns. OpenBMB marks SciCode 26.3, Terminal-Bench v2.1 8.6, GDPval-AA v2 19.6 and several other rows with the dagger marker indicating Artificial Analysis provenance. This is a better disclosure pattern than presenting the whole table as one internally controlled evaluation, but readers still need to inspect provenance row by row.

OpenBMB claims its 53.9 aggregate leads the selected 2B comparison set and even exceeds the larger reference models included in that specific table. That is a vendor-defined comparison set, not proof that MiniCPM5-2B beats every 3B–4B model under every independent benchmark, harness or practical workload.

Training-data openness is a real strength

OpenBMB says the training pipeline covers base training, mid-training and post-training. The release links the Ultra-FineWeb, Ultra-FineWeb-L3, UltraX, UltraData-Code and UltraData-Math corpora, plus SFT and RL datasets.

For post-training, OpenBMB reports 400 billion tokens of deep-thinking SFT, specialized RL teachers for mathematics, code, agentic tasks, writing and related domains, followed by On-Policy Distillation. It says OPD merges capabilities from 16 expert models, including five agentic experts, and reports average gains of 10.96 points on its listed reasoning/general benchmarks and 6.96 points on the listed agentic benchmarks over the preceding stage.

Those improvement figures are again vendor-run ablations. The important reproducibility advantage is that OpenBMB is releasing multiple training stages and substantial datasets, giving outside researchers more material to audit than a final checkpoint alone.

Pricing and latency: open weights does not mean zero compute cost

Artificial Analysis displays $0 input / $0 output token pricing for MiniCPM5-2B because the model is open weight and self-hostable; it currently lists no cost-per-Intelligence-Index-task measurement and no output-speed result. Those zeros should not be interpreted as free inference. Self-hosting still consumes GPU, CPU, memory, electricity and operator time, and a commercial host can charge its own rate.

The Hugging Face model page currently says MiniCPM5-2B is not deployed by any Hugging Face Inference Provider. OpenBMB documents local serving with vLLM, SGLang and Transformers, and also provides GGUF, MLX and GPTQ variants. Without a standardized first-party hosted endpoint, there is no durable list price, TTFT or output-token-per-second number that can fairly characterize the model across hardware.

OpenBMB also publishes a DSpark draft model for speculative decoding. That is a useful deployment feature, but no single wall-clock speedup should be promised without a pinned hardware, batch size, context length, quantization and decoding setup.

Context and modality tradeoffs

The official card specifies a 131,072-token context window, while Artificial Analysis reports approximately 131K. MiniCPM5-2B is text input and text output only; it is not a vision-language model. Developers comparing it with small multimodal systems should therefore separate language/agent performance from image understanding.

Artificial Analysis lists a December 31, 2025 knowledge cutoff in its model metadata. That is useful third-party metadata, but OpenBMB's launch card does not prominently state the same cutoff in the main technical specification block, so it should not be presented as an independently verified training-data boundary from OpenBMB.

Public feedback is too early for a reliable consensus

A bounded launch-day search of accessible X and Reddit results did not surface a current, controlled MiniCPM5-2B reproduction with pinned weights, quantization, hardware, prompts and benchmark traces. Older community discussion around the MiniCPM5 family exists, including local-model users discussing the earlier 1B model, but those anecdotes cannot be transferred to this new 2B checkpoint.

That means claims such as “best local 2B model in real use,” “faster than every competitor,” or “better than a 4B model for coding” are still premature outside OpenBMB's selected launch table. Launch-day community interest is evidence of attention, not measured consensus.

Practical take

MiniCPM5-2B is a notable compact release because it combines a 2.516B dense architecture, 131K context, Apache-2.0 weights, multiple quantized formats, released training stages and substantial public training data. Its provider-reported 46.4 SWE-bench Verified and 14.4 SWE-bench Pro results are strong enough to merit independent reproduction, especially at this parameter scale.

The independent evidence is more conservative but still positive: Artificial Analysis gives it 15 on Intelligence Index v4.2 and #1/47 in its tiny open-weight class. That score should not be compared numerically with OpenBMB's 53.9 launch average, and neither should be used to collapse SWE-bench Verified, SWE-bench Pro and Terminal-Bench into one ranking.

For developers, the best next step is a pinned local evaluation on the exact final BF16 or declared quantized checkpoint, recording hardware, runtime version, context, reasoning mode, tool harness, latency, token usage and success criteria. For benchmark claims, the next evidence to watch is an independent SWE-bench Verified/Pro rerun with exact harness details and public trajectories.