Tencent EVIE Reality Check: 66.75 Leads ViDoRe V3, but the 3.81 GiB Index Scores 59.58
Tencent's EVIE-8B reports 66.75 on ViDoRe V3 and EVIE-4.5B 66.02, but the smallest 3.81 GiB-per-million-page HAC index scores 59.58. The release is open and well documented, while independent final-checkpoint verification and pinned latency remain open.
What Tencent released
Tencent's current EVIE-8B and EVIE-4.5B checkpoints are open-weight visual document retrieval models, not general chat models. The repositories were actively updated on September 7, 2026 and are published under Apache 2.0. EVIE-8B uses a Qwen3.5-9B backbone with 8.41B parameters and 4096-dimensional per-token representations; EVIE-4.5B is the smaller Qwen3.5-4B-based student with a single elastic Prefix-MRL projection that can be truncated from 2048 dimensions down to 64 dimensions at runtime.
Unlike a text generator, EVIE turns a query and document-page images into multi-vector representations and scores relevance with late-interaction MaxSim. The intended use is document search and retrieval for visually rich pages containing tables, charts, typography and layout.
Primary sources:
The 66.75 ViDoRe V3 result is strong, but it is still Tencent's evaluation
Tencent reports 66.75 nDCG@10 on ViDoRe V3 for EVIE-8B and 66.02 for EVIE-4.5B. In Tencent's comparison table those are the highest two V3 values shown, ahead of the listed webAI, Vultron, Nemotron and Tomoro retrievers.
The useful part is that Tencent publishes substantially more methodology than a bare leaderboard screenshot. Its 138-task matrix identifies the protocol as paired-all-pages-dedup+process_queries+ndcg2r-20260827, with maximum visual tokens set to 1024 and bidirectional attention. The matrix covers:
- ViDoRe V1: 10 tasks, 3,943 queries and 6,729 documents
- ViDoRe V2: 4 tasks, 1,152 queries and 4,544 documents
- ViDoRe V3: 48 tasks, 14,514 queries and 115,512 documents
- JinaVDR: 76 tasks, 43,375 queries and 55,398 documents
That makes the result more reproducible than an unexplained score. It does not turn it into an independent result. In this review I did not find an official ViDoRe/MTEB leaderboard record that independently pins the exact final EVIE checkpoint, revision and submitted run. Until such a record or a third-party rerun is available, 66.75 should be labeled vendor-reported.
ViDoRe V3 itself is a meaningful benchmark: the benchmark authors describe 10 enterprise-oriented datasets, more than 26,000 pages, 3,099 base queries translated into six languages, eight public and two private datasets, and human-created/human-verified annotations. That strengthens the benchmark design, but it is separate from independently validating Tencent's EVIE submission.
Sources:
“Rank #1” needs benchmark-specific reading
Tencent's V3 claim is straightforward inside its own table: 66.75 is the highest V3 nDCG@10 value listed.
The broader “rank #1” presentation needs more care. In the same comparison table, the ViDoRe V1 nDCG@5 column gives EVIE-8B 92.18, while nemotron-colembed-vl-8b-v2 is listed at 92.65 and VultronRetrieverCore-4.5B at 92.21. Tencent's GitHub badge also labels “ViDoRe V1+V2 — 92.18 · Rank #1,” even though 92.18 is the displayed V1 nDCG@5 value rather than a clearly documented V1+V2 aggregate in that table.
The practical conclusion is not that EVIE's results are invalid. It is that rank claims should be attached to the exact benchmark, metric and operating point. “Highest in Tencent's displayed ViDoRe V3 table” is supportable; “universally best visual retriever” is much broader than the evidence.
The 3.81 GiB index and the 66.02 score are not the same operating point
EVIE-4.5B's most useful engineering feature is its elastic representation. With no separate model, Tencent reports the following ViDoRe V3 scores as the projection is truncated:
- 2048D: 66.02
- 1024D: 65.94
- 512D: 65.90
- 256D: 65.68
- 128D: 65.27
- 64D: 64.51
The model card then adds training-free Hierarchical Agglomerative Clustering (HAC) to reduce the number of visual vectors stored per page. This is where a headline can become misleading if two different configurations are combined.
Tencent's smallest published production SKU, d64 K32, uses 64-dimensional vectors and only 32 vectors per page. It is listed at 4 KiB per page, 3.81 GiB per one million pages, but its ViDoRe V3 score is 59.58, not 66.02. The d64 K64 configuration uses 64 vectors per page, takes 7.63 GiB per million pages and scores 62.06 on V3. The d128 K32 configuration also uses 7.63 GiB and scores 61.40.
So the impressive 3.81 GiB storage figure is real according to Tencent's table, but it carries a 6.44-point V3 drop versus the full 2048D EVIE-4.5B result. Teams choosing EVIE should benchmark the actual dimension and HAC setting they plan to deploy rather than combining the best accuracy number with the smallest index number.
SWE-bench Verified and SWE-bench Pro are not applicable here
EVIE is a visual document retriever, not a software-engineering agent. I found no exact EVIE-8B or EVIE-4.5B SWE-bench Verified result and no SWE-bench Pro result. That is expected and should not be treated as a missing coding capability claim.
Scores from the underlying Qwen backbones must not be imported into EVIE. Fine-tuning, bidirectional retrieval architecture, projection heads and the task itself are different. For this release, ViDoRe and JinaVDR are the relevant benchmark families; SWE-bench Verified and SWE-bench Pro remain explicitly not reported/not applicable.
Pricing, latency and access
The weights and code are available under Apache 2.0, so there is no single first-party per-token EVIE tariff. Deployment cost depends on the GPU stack, document volume, chosen embedding dimension, visual-token budget and index configuration.
At verification time, the checked Hugging Face model pages did not expose a hosted Inference Provider endpoint for these checkpoints. Tencent provides ColPali Engine and Sentence Transformers examples, but I did not find a controlled, version-pinned latency benchmark that would justify publishing a universal requests-per-second, TTFT or indexing-throughput number. Those values should therefore remain unknown until measured on the target hardware and corpus.
For production RAG, the more actionable published economics are currently storage and representation size: 4096D for the 8B teacher, elastic 64D–2048D for the 4.5B student, and HAC choices down to 32 stored vectors per page.
Public feedback: practitioners immediately noticed the same tradeoff
A September 7 r/LocalLLaMA thread sharing the final EVIE-8B and EVIE-4.5B releases attracted early interest in the compact index and visual retrieval use case. One commenter specifically pointed out that the 3.81 GiB configuration scores 59.58 on V3 rather than 66.02, which is the same operating-point distinction visible in Tencent's own table.
That is useful feedback because it identifies a likely source of misunderstanding, but it remains self-selected community commentary, not an independent benchmark. The thread does not control hardware, corpus, checkpoint revision or retrieval settings, and it should not be used to infer consensus about quality.
I did not accept an X quote in this bounded review because I did not retrieve a stable, directly attributable x.com/.../status/... post that added benchmark or deployment evidence. No quote or “community consensus” was invented.
Practical verdict
EVIE is a technically interesting open visual-retrieval release. Tencent publishes the weights, code, evaluation protocol, training/distillation material and unusually clear storage-versus-accuracy tables. EVIE-8B's 66.75 ViDoRe V3 and EVIE-4.5B's 66.02 are strong vendor-reported results, and the smaller model's Prefix-MRL design gives deployment teams a useful continuum between quality and index size.
The biggest caution is simple: do not combine numbers from different configurations. The full 2048D EVIE-4.5B reaches 66.02 on V3, while the 3.81 GiB-per-million-pages d64 K32 configuration reaches 59.58. And “rank #1” should be attached to the exact V3 metric rather than generalized across every ViDoRe column.
The next high-value evidence would be an independent, version-pinned EVIE submission on the official ViDoRe/MTEB leaderboard plus reproducible indexing/query latency on named hardware at several Prefix-MRL and HAC operating points. That would make it possible to judge not only retrieval quality, but also the actual quality-per-dollar and quality-per-gigabyte tradeoff.
This article is built from the source material below. Open the originals for full context and the latest updates.
