NeoMME Reality Check: 0.523/0.556 ViDoRe v3, 51 Pages/s and 255× Compression

What NeoMME actually is

On September 3, 2026, H Company researchers Aurélien Lac and Tony Wu published NeoMME, a family of 260M- and 800M-parameter multilingual, multimodal bidirectional encoders. The models are not chat assistants and they are not autoregressive vision-language generators. Text tokens and raw image patches go through one shared Transformer encoder, with no separately pretrained vision tower and no causal language-model decoder.

The base NeoMME checkpoints return contextual representations and need a task-specific head for retrieval, classification, extraction or another downstream task. For document search, the relevant released checkpoints are NeoMME-Retriever, which add jointly trained dense and late-interaction retrieval heads.

Both sizes support a 16,384-token context. H Company says that is enough for up to two standard 3840×2160 4K UHD images, although the released model cards use a default maximum image side of 2,048 pixels. Images are split into non-overlapping 32×32 patches and share the same transformer path as text.

The 0.523 and 0.556 ViDoRe v3 scores are promising, but they are author-run

H Company reports the following late-interaction results on ViDoRe v3 nDCG@10:

• NeoMME-Retriever-260M: 0.523 (the model card gives 0.5226)
• NeoMME-Retriever-800M: 0.556 (0.5560 on the card)

The release post says the 260M checkpoint is the highest-scoring evaluated model strictly below 800M parameters and is within roughly 0.002 nDCG@10 of ColQwen2.5 while using about 14× fewer parameters. It places the 800M model within 0.009 of the similarly sized Vultron Retriever Flash.

The important methodology caveat is in the same table: NeoMME's numbers are marked as results from the authors' own evaluations, while several comparator values are taken from MTEB. That means the chart mixes result provenance. It is useful evidence from the release team, but it is not yet an independent reproduction under one frozen third-party harness.

That matters because tiny score gaps can be smaller than differences caused by model revision, preprocessing, image resolution, batching, library versions or evaluation implementation. The fair conclusion is not “NeoMME independently beats every model in its size class.” The fair conclusion is that the authors report a strong size-versus-retrieval-quality tradeoff that now needs external replay.

What ViDoRe v3 measures

ViDoRe v3 is a visual-document retrieval and RAG benchmark, not a general intelligence or coding benchmark. Its paper describes 10 datasets, roughly 26,000 document pages, and 3,099 human-verified queries, with queries available in six languages. The benchmark required about 12,000 hours of human annotation and includes relevance labels, bounding boxes and reference answers.

The ViDoRe team also says eight datasets are public and two are private. That is important context when comparing reproduced public results with leaderboard or author-reported numbers: a score is only directly comparable when the same dataset subset, metric and evaluation setup are used.

NeoMME's headline table reports nDCG@10 for ViDoRe v3. Older ViDoRe v1 and v2 results in the same release use nDCG@5, so those numbers should not be ranked directly against v3 as though they were one common score.

Late interaction is much stronger than NeoMME's dense head on the reported retrieval tests

The current NeoMME-Retriever model card makes the dense-versus-late-interaction tradeoff visible. On ViDoRe v3, the 260M model reports 0.5226 late-interaction versus 0.3907 dense, while the 800M model reports 0.5560 versus 0.4391.

On the model card's BEIR-15 text-retrieval aggregate, the reported values are 0.4881 versus 0.3055 for 260M and 0.5126 versus 0.3686 for 800M.

This does not mean dense retrieval is useless. Dense embeddings are much smaller and fit standard approximate-nearest-neighbor indexes. H Company explicitly suggests a practical two-stage design for large corpora: use the dense embedding to retrieve a candidate set, then rerank those candidates with the more expressive late-interaction vectors.

The benchmark lesson is therefore architectural: NeoMME's strongest reported retrieval quality comes from the late-interaction head, not from the compact single-vector representation alone.

51 pages per second is an indexing-throughput result, not universal latency

At a matched 2048×2048 page-image input size on one NVIDIA L40S, H Company reports that NeoMME-Retriever-260M encodes about 51 pages per second, compared with about 26 pages per second for ColModernVBERT.

The release says preprocessed image tensors were used and batch size was calibrated separately for each model and image size. Those details are critical. Throughput measured with tuned batching is not the same thing as per-request latency, and it should not be converted into a universal milliseconds-per-page number. Real systems also pay for PDF rasterization, data transfer, queueing, vector-store writes and query-time search.

So 51 pages/s is evidence about one author-run indexing configuration on one GPU and one matched resolution. It is not a cross-cloud SLA or a guaranteed interactive latency.

The 255× compression result is also conditional

Late-interaction retrieval normally produces many vectors per page. H Company says a 2048×2048 page produces around 4,200 vectors, about 2.1 MB in float32, and that the average across its ViDoRe v3 measurement is around 1.5 MB per document.

The team combines hierarchical token pooling with asymmetric quantization. It reports two useful operating points for NeoMME-260M:

• pooling factor 10 with int8 queries and documents: about 39 kB/page, roughly 39× smaller, while retaining more than 99% of baseline nDCG@10;
• pooling factor 8 with int8 queries and binary documents: about 6 kB/page, roughly 255× smaller, while retaining more than 95% of baseline nDCG@10.

“255× smaller” therefore does not mean lossless compression. It is a measured point on a storage-versus-retrieval-quality curve, and the retained quality is stated relative to the team's own baseline on ViDoRe v3. Production corpora with different document layouts, languages or relevance distributions can land elsewhere on that curve.

Training scale and architecture are unusually small for a multimodal foundation encoder

H Company says each NeoMME size processed about 524 billion packed input tokens, including roughly 290 billion text-only tokens. The pretraining mix includes multilingual text, code, mathematics, natural images and document images.

The training objective is a masked discrete-diffusion text denoiser rather than autoregressive next-token prediction. On multimodal examples, image patches stay visible while the model reconstructs masked text. Higher text corruption forces more reliance on visual evidence.

This setup helps explain what the model is optimized for: compact, bidirectional representations of text and documents rather than free-form generation. Users who need an answer-writing VLM still need a separate generation model after retrieval.

Open weights do not mean a hosted API with a known per-token price

The checkpoints are released under the Apache 2.0 license and are integrated with Hugging Face Transformers and Sentence Transformers. The 800M base-model card currently says it is not deployed by a Hugging Face Inference Provider.

That means there is no single verified NeoMME-specific hosted API price comparable with the per-token price of a commercial chat model. The practical cost depends on hardware, precision, batching, image resolution, index format and serving stack. “Open source” tells you about licensing and deployability; it does not make inference cost zero.

The same model card also states that NeoMME has not received a comprehensive safety, bias or privacy evaluation. That limitation matters for sensitive document collections, especially when multilingual or scanned material contains personal data.

SWE-bench Verified and SWE-bench Pro are not applicable scores

NeoMME is a multimodal representation encoder and document retriever. It is not a repository-editing coding agent. Therefore:

• SWE-bench Verified: not an applicable NeoMME benchmark result.
• SWE-bench Pro: not an applicable NeoMME benchmark result.

A visual retrieval score cannot be translated into a software-engineering pass rate, and a coding benchmark would not be a meaningful substitute for ViDoRe retrieval quality unless NeoMME were embedded inside a larger coding system with a clearly specified role.

Public feedback is too thin for a reliable sentiment claim

I did not find a sufficiently attributable, NeoMME-specific public discussion on X or Reddit that both exposed the original post and provided enough context to treat it as meaningful user feedback. H Company's launch article links an X explainer about the general terminology of late interaction, but that is not a NeoMME review, and the X page was not publicly fetchable in this verification pass.

Hugging Face likes, downloads and collection activity show that people are inspecting the release, but they are not reviews and should not be converted into “community consensus.” Until reproducible third-party tests appear, the strongest evidence remains the released weights, model cards, paper and benchmark definitions.

The practical tradeoff

NeoMME is interesting because it attacks visual-document retrieval from the opposite direction of ever-larger generative VLMs. The architecture is compact, the weights are open, one forward pass can provide both dense and late-interaction embeddings, and the author-reported results show a strong quality/size/throughput frontier.

The limitations are equally clear. The headline retrieval scores and throughput are currently author-run, comparator provenance is mixed, the 255× compression point accepts measurable quality loss, there is no official hosted price, and there is not yet enough independent public feedback to claim a consensus.

For teams building visual RAG, NeoMME is therefore best viewed as a promising open retriever worth benchmarking on their own documents—not as a universal replacement for OCR, dense retrieval, or generative multimodal models.