Hands-On: Perplexity pplx-embed-v2-late — the 0.6B Model That Queries a 9B Index

pplx-embed-v2-late is a late-interaction (ColBERT-style) retriever: a document is not compressed into one vector; token-level vectors are kept, and at query time each query token takes its highest similarity against document tokens and the maxima are summed (MaxSim). The models are built on Qwen3.5 with bidirectional attention. Both sizes were distilled from one 18B teacher, trained on 186 million query-document pairs in 46 languages: 88.3% text-to-text, 8.3% text-to-image and 3.4% text-to-visual-document.

The 0.6B model has 594 million parameters in total, with about 240 million active for text encoding and 340 million for image encoding; the 9B activates 7.4 billion. Perplexity states the two sizes share one embedding space: an index built with the 9B can be queried directly with 0.6B-encoded queries. Weights were downloadable on release day; a hosted API is not yet live, and a full technical report is planned for later this year.

1. Published scores (Perplexity blog, Oct 7, 2026)

Visual-document retrieval on ViDoRe v3 (public subset, nDCG@10): image retrieval 62.3% for the 0.6B and 65.2% for the 9B; on the Markdown-converted text version, 61.2% and 64.7%, with the 0.6B ahead of every external model on that task and behind only the 9B. On 72 domain-specific text tasks the averages are 78.0% (0.6B) and 81.3% (9B), the 9B leading the next evaluated model by 1.6 points. On web-scale Q2D-Web (about 70,000 queries over 190 million documents, Recall@1000, Combined judgments): 73.6% (0.6B) and 74.8% (9B), against a previous best of 69.3%.

On agentic QA benchmark MADQA (800 PDFs, over 18,000 pages, 500 human-authored questions; agent: Gemini 3.5 Flash; scored by answer accuracy): 92.4% for the 9B and 90.1% for the 0.6B, versus 88.9% for the same agent paired with Mixedbread's retriever. On BrowseComp+ (agent: GPT-OSS-120B) the 9B reaches 64.0% answer accuracy, 4.9 points ahead of the next ColBERT model. Cross-size retrieval — 9B index, 0.6B queries — scores 63.5% on ViDoRe v3 image retrieval (versus 62.3% for 0.6B on both sides) and adds 1.6 points on the domain-task average; Perplexity states this setup recovers about half of the text quality gap between the two sizes at unchanged query-encoding cost. All figures in this part were published by Perplexity; the technical report is not out and no third-party replication is available yet.

2. Local hands-on test (0.6B, CPU)

Setup: a Linux machine with 2 CPU cores and 7GB of RAM, no GPU; current sentence-transformers 6 and transformers 5 releases, weights downloaded from Hugging Face and loaded locally. The corpus is 20 short passages written for this test (16 English, 4 Chinese, non-overlapping topics), with 12 queries (9 English, 3 Chinese), each matching exactly one passage. Results: Recall@1 100%, Recall@3 100%, mean reciprocal rank (MRR@10) 1.00. Encoding the 20 documents took 13.4 seconds; query encoding plus scoring averaged 0.7 seconds per query (CPU conditions only, not representative of GPU or production serving); peak process memory was about 1946 MB.

One cross-modal check: a self-made quarterly-results page image (product name, units shipped, defect rate) was added to the corpus and searched with text queries. 3 of 3 test queries ranked as expected: the two queries asking about data on the page ranked it first, and an unrelated query did not. With a sample of 3, this is a functional check only, not an accuracy claim.

Limits of the local test: the 9B weights were not loaded on this machine's memory budget, so no local 9B numbers exist here; the local corpus is self-written short passages, easier than the real documents in ViDoRe and MADQA, and its hit rates are not directly comparable with the published scores. The score below rests on three parts: the published benchmark results, the published cross-size data, and the local results above.

1. Release: Oct 7, 2026; weights live on Hugging Face the same day under the MIT license, commercial use permitted.
2. Architecture: Late interaction (ColBERT-style), one 128-dimensional vector per token, MaxSim scoring; a single input cannot mix text and images — they are encoded in separate calls.
3. Storage cost: Vector count grows with document token count, so long-document indexes are larger than single-vector ones; the visual models compared in the official post use 2,048–4,096 dimensions per vector, against 128 per token here.
4. Access: Self-hosted download; requires sentence-transformers 6.0.0 and transformers 5.4.0 or newer; Perplexity's hosted API is not yet live.
5. Open items: The full technical report is planned for later this year; the published scores have no third-party replication yet.