How Much To Run AI
Open weights

Sarvam Prism: Decision API Costs and Self-Hosting

Sarvam Prism selects among known answers for routing, scoring and yes/no decisions. It returns probabilities rather than generated prose. Use Sarvam’s hosted Decisions API to avoid GPU setup, or run the open-weight model with its custom inference code on a suitable FP8-capable NVIDIA GPU.

Content reviewed:

Use this model

API docs and console

Official model repository

Use cases

  • Route a support ticket to one of your defined teams.
  • Score a recorded agent run against a fixed rubric.
  • Send uncertain decisions to review using thresholds checked on labeled examples.

What does Sarvam Prism return?

Prism is built on Sarvam 105B and scores a fixed answer set in a single forward pass. Use it when your application already knows the possible outcomes. For a free-form explanation or a new document, choose a text-generating model instead.

The hosted endpoint accepts a shared state and named questions. Use noul for yes/no, choice for categories and score for ordered levels. A probability is relative to the supplied options: design those options carefully and evaluate the decision against labeled cases.

Sarvam Prism API pricing

Sarvam lists ₹4 per million input tokens and no output-token charge, excluding applicable GST. The state is shared across questions and billed once per request; each question’s instructions and options add input tokens. Read usage.input_tokens to calculate the actual bill.

An illustrative request with 1,000 billable input tokens costs ₹0.004. Ten thousand such requests total 10 million tokens, or ₹40 before tax. Longer state, additional questions and retries increase this budget.

Call Prism with state and typed questions

Install sarvamai version 0.1.43 or later, create an API key and store it as SARVAM_API_KEY on the server. The SDK’s decisions.create method calls the Decisions endpoint, rather than a chat-completions interface.

Inspect the returned answers and per-option probabilities. Use a labeled validation set to choose your escalation threshold; a high model confidence does not replace application-level quality checks. Keep the original input and decision together for review.

Illustrative SDK request; sending it incurs API usage charges
import os
from sarvamai import SarvamAI

client = SarvamAI(api_subscription_key=os.environ["SARVAM_API_KEY"])
response = client.decisions.create(
    model="sarvam-prism",
    state={"ticket": "Checkout payments fail after the latest release."},
    questions={"team": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {
            "engineering": "Software bugs and outages",
            "billing": "Invoice and refund questions",
            "sales": "New business enquiries",
        },
    }},
)
print(response.answers)

Hosted input, context and timeout limits

The hosted API documentation specifies a 32,768-token limit per question, counting the shared state and that question’s instructions and options. Oversized input fails rather than being silently truncated. Keep this limit separate from the local model card’s larger context figure.

The API takes text or JSON state, returns synchronous JSON and does not accept images or stream a rationale. Choice supports 2–255 options; score supports 2–10 ordered levels. Split large question sets to fit the documented 60-second request timeout. Check your account’s current rate limits before planning a batch.

What hardware does self-hosted Prism need?

The official model card requires an FP8-capable NVIDIA GPU with about 120GB free memory, about 106GB disk, Python 3.10 or later and a C compiler for Triton kernels. Its custom FP8 layout uses the supplied inference code; a generic AutoModelForCausalLM or standard chat server is not the documented launch path.

The weights and code are Apache 2.0. Follow the repository’s quick start and retain its notices when redistributing. Compare the cost of reserved accelerator time and operations with the hosted input bill before choosing a local deployment. The 105B family label alone does not tell you the local memory requirement.

How to read Prism’s speed and quality claims

Sarvam reports a 71.8 average across three decision indices in its launch comparison. One index is internal and another is its recreation of an external index; these are publisher-run evaluations. Its 44ms median latency is a B200 inference measurement after loading, rather than the complete network and application workflow.

Build an acceptance set for your actual labels and languages. Measure accepted decisions, review rate and end-to-end latency under the traffic you expect. Compare Prism with Jev or OpenAI Decisions when the task has fixed outcomes; compare a chat model when users need prose explanations.

Before you use it

  • It selects from supplied outcomes and does not generate an explanation. Use a separate explanation step when the user needs one.

How to get started

Define the allowed outcomes, validate thresholds on your own cases, then compare hosted token cost with the complete self-hosted service budget.

Sources

Sources (5)

Calculators