EmbeddingGemma 2: Local Retrieval, RAG and Vector Costs

The text and code backbone has 270M parameters. Adding vision brings it to 440M; adding audio to text brings it to 570M; all modalities total 740M. The model shares an 8,192-token context and returns 768-dimensional embeddings, with supported truncation to 512, 256 or 128 dimensions.

Google’s developer guide lets you disable unused encoders at loading time. A text search service can start with the 270M path instead of loading image and audio weights. Rough weight-only arithmetic at two bytes per parameter is 540MB for text and 1.48GB for all modalities. Activations, input processing, the runtime and the vector index add memory beyond those numbers; allow room for those components when sizing the device.

Use bf16 or fp32 for model inference. The model card warns that fp16 can produce NaNs or degrade results silently. A database’s float16 vector storage is a separate choice from running model weights in fp16. Test the model’s output vectors for finite values and retrieval quality before building the full collection.

Install the Sentence Transformers and Transformers versions using pip install -U sentence-transformers transformers, then load the official checkpoint. This example follows Google’s text-only loading path. SearchQuery and Document select the retrieval prompts: use the query prefix for a question and the document prefix for indexed material.

Keep the same dimension for query and document vectors. When shortening vectors, truncate first and normalize the result again to unit length. The example does both through the encode options. Record the checkpoint, prompt configuration, dimension and precision with the collection so a later client cannot silently query it with incompatible vectors.

Google reports a full-precision Code MTEB score of 78.68, compared with 68.76 for the first generation. Multilingual MTEB is 61.36 versus 61.15. The stronger code result should not be generalized into a uniform improvement for every language, corpus or retrieval task.

Lower dimensions trade space for recall. In Google’s reported multimodal evaluation, the MMEB score drops from 59.01 at full dimensions to 45.65 at 128 dimensions. Start with 768 or 512 for visual documents or mixed modalities, then compare your labeled queries before reducing dimensions. A dimension that works for short text can lose details needed to match an image or audio clip.

A retrieval benchmark checks which evidence is found. It does not measure the accuracy of a generated answer, its citations or the safety of an action taken afterward. Google recommends pairing the model with Gemma 4 for on-device RAG. Check complete answers and citations against your documents.

Ten million 768-dimensional float32 vectors contain 30.72GB of numeric data: 10,000,000 × 768 × 4 bytes. Qdrant’s 768-dimensional 1-bit TurboQuant representation uses 104 bytes per vector, or 1.04GB for the same vector count. Its 256-dimensional 1-bit representation uses 40 bytes, or 0.40GB. These are calculated vector-only sizes, before graph indexes, document payloads, replicas and any original vectors retained for rescoring.

Qdrant evaluates text retrieval on SciFact, NFCorpus, ArguAna, SCIDOCS and FiQA. Its separate latency experiment uses Quora’s 523K documents; that is not the combined document count of the five BEIR datasets. Across the text-quality tests, 768d 1-bit retains about 99% of nDCG@10, 512d 1-bit about 97.3%, and 256d 1-bit about 94.5% with 4× rescoring. Without rescoring, the 256d result is 88.1%.

Those results concern stored text vectors. They do not quantify multimodal retrieval, model-weight quantization or the quality of a full RAG answer. Leaving rescoring off avoids reading original vectors during a query; it does not by itself delete those vectors from storage. Include the actual collection configuration in a cost comparison.

A RAG pipeline starts by chunking source material, embedding each chunk and storing its vector with a stable source identifier. Each question then needs a query embedding, a search and a generation call using the selected chunks. Keep document access rules attached to retrieval so evidence is filtered before it reaches the answer model.

First-time indexing processes the whole corpus. Incremental indexing processes added or changed chunks. Moving from the first EmbeddingGemma to version 2 changes the embedding space and requires re-embedding the corpus into a new collection. Within version 2’s shared space, Google says enabling an additional modality encoder does not require recomputing existing embeddings. A dimension or task-prompt change still needs a compatible collection and query configuration.

For API generation, Haiku 5.5 is an option for a bounded extraction or short answer from retrieved evidence. For downloadable generation models, compare Qwen3.8 27B and Mistral Large 4 on the same evidence and accepted-answer checks. EmbeddingGemma finds material; the generation model must still use it correctly. Measure retrieval misses separately from answers that ignore a relevant retrieved chunk.

Track indexing compute time, refresh frequency, vector and payload storage, query latency and generator input/output tokens. Compact vectors can reduce a storage bill while a frequently refreshed corpus remains expensive to embed. Sending fewer useful chunks can also reduce the generation bill, but confirm that the answer still has the necessary evidence.

Embedding Gemma Transcribe is an independent Mac app that pairs the unchanged audio encoder with a trained reader for speech recognition. The reader maps audio slices into English words and a small model adds punctuation. This application extends the embedding model into a transcript workflow; the base checkpoint still returns embeddings.

The author reports transcribing one hour of English audio in 12.6 seconds on an M3 MacBook Air with 8GB memory, using the Neural Engine. This is the app author’s timing, with its reader and device configuration. Its first launch downloads 715MB of models; that download size is not a RAM requirement. The app requires macOS 14 or later and Apple Silicon, and processes speech on the device.

The Google checkpoint’s Apache 2.0 license does not establish the license or availability of this app’s trained speech reader. Use the app download for this workflow; keep the base model and the application’s release terms separate.