pplx-embed-v2-late: Retrieval and Index Storage Costs

The models use a Qwen3.5 backbone with bidirectional attention and return 128-dimensional vectors per token. MaxSim compares each query vector with document vectors, takes its best match, then combines the matches into a retrieval score. A result points to source material; a separate generator writes the answer.

The cards list 340M active parameters for 0.6B and 7.4B for 9B. The 0.6B figure includes the vision path. Budget memory for the complete checkpoint and its input batches. Both use MIT licensing. Keep the checkpoint and encoder configuration with the collection. The family’s shared space supports 0.6B queries against 9B documents; using the smaller query encoder does not shrink an already built 9B index. Budget document encoding for both the initial collection and updates; unrelated embedding models cannot query that index interchangeably.

For a first pplx-embed-v2-late search, install sentence-transformers>=6.0.0 and transformers>=5.4.0, then use MultiVectorEncoder. The author’s usage example below encodes one query and two passages; model.similarity performs MaxSim. Choose a device that can load your checkpoint and leave room for input batches.

For a PDF, render a page to an image and pass a PIL RGB image to encode_document. Keep page numbers and the original file alongside the vectors so a search result can open the exact source page. Text-only and image-only batches need separate encoding calls; mixed text-and-image batches are unsupported. PyLate’s default marker placement differs from this model’s expected first-position markers. Both exported wrappers set query_length to 1024 and document_length to 4096; a backbone position limit is not the out-of-box retrieval input budget. Inspect truncation when preparing long passages.

For a pplx-embed-v2-late index, raw vector storage equals document count × mean retained vectors per document × 128 dimensions × bytes per value. With an assumed 100,000 documents and 256 retained vectors each, float32 vectors use 13.11GB in decimal units; two-byte storage would use 6.55GB. These estimates exclude IDs, metadata, originals, index structures, replicas and working memory. Verify retrieval quality before changing storage precision.

Visual-page token counts depend on image processing. Measure the actual vector count for a representative PDF sample instead of multiplying a text tokenizer count by page count. Tables, charts, scans and unusually large pages should all be in that sample.

Indexing is paid whenever new or changed content is encoded. Deduplicate unchanged pages and record content hashes so a small update does not trigger a full rebuild. A blue-green replacement index needs capacity for old and new collections at once; include that temporary peak in the storage plan.

As of October 9, 2026, Perplexity’s public Embeddings API quickstart lists four v1 dense and contextualized models, with no v2-late endpoint or rate. Use the downloadable v2-late checkpoints for the workflow described here. The v1 token prices apply to different representations and cannot price this multi-vector index.

For self-hosting, add page rendering, document encoding, persistent storage, query encoding and MaxSim search. At an illustrative $1 per GPU-hour, a six-hour indexing job costs $6; keeping a dedicated GPU service provisioned for 720 hours costs $720 before storage and other services.

The 9B-index/0.6B-query arrangement moves heavier encoding to the offline job. It does not remove multi-vector search work. Measure tail query latency with your candidate count and index implementation, and include recurring index updates in the monthly total.

To evaluate pplx-embed-v2-late retrieval quality, build a labeled set of questions with relevant page IDs. Compare 0.6B on both sides with 9B documents and 0.6B queries using the same corpus, candidate budget and scoring pipeline. Record Recall@k, answer-supporting pages found, query latency and index size. Keep errors such as a correct table on the wrong page visible in the review.

Perplexity reports public ViDoRe v3 image nDCG@10 of 62.3% for 0.6B and 65.2% for 9B. Those are publisher results on that visual retrieval benchmark. They help select a trial configuration; the relevant deployment result is whether your own questions retrieve the evidence required for a correct answer.

After retrieval, send selected pages or extracted evidence to a generator. Choose a text generator for extracted passages and a vision-capable generator when the answer depends on the original page image. Keep the source citation with the answer and check that it supports the statement.