Open-Source Decision Model Cost Calculator

When should you use a decision model?

Use a decision model when the answer is a finite label, a score or a yes/no judgment, and your program already knows what to do with it. Examples include ticket → billing, request → cheaper model, email → spam probability, or document → relevance score.

It can also be the first filter: reduce 100,000 documents to a shortlist for an LLM, send urgent tickets to a person, or escalate suspicious agent outputs for a more expensive review. Keep the accepted labels and the uncertain branch explicit.

Choose a generative model when the result itself is the product: an article, explanation, summary, research report, code or an open-ended plan. A decision model can route that work; it does not create the deliverable.

Keep arithmetic, dates, numeric thresholds and permission enforcement in code. Test confidence on representative cases instead of treating 0.8 as 80% accuracy. Define what happens when the model is uncertain, times out or returns an invalid answer.

Turn a workflow into explicit questions

Support triage: choose billing, technical support or account access; score urgency on a defined 1–5 scale; check whether a refund is requested. These are separate questions. Route the ticket or ask a person to review it; the model does not authorize the refund.

Review and guardrails: check whether an answer is grounded in supplied material, meets a rubric or needs human review. Calibrate review thresholds with false positives and missed violations. Use a generative model when you also need an explanation.

RAG filtering: score each candidate document against the request, shortlist the relevant material, then let a generative model read it. Validate recall so that a cheap first pass does not discard the evidence needed for the final answer.

What are people building with decision models?

These projects show how a bounded judgment connects to working software. The results below are their authors’ reports, with the task and operating conditions kept alongside the numbers.

Published decision model latency references

The Laya project reports these Tesla T4 timings. The table does not pin a complete runtime/checkpoint revision and workload specification for every row. These are reference figures, not selectable measured profiles or results measured by this site.

No Typed Decisions timing is inferred from the English checkpoint. CPU, ONNX, fast-path and other GPU setups need their own measurements. Parameter count and weight size do not determine request throughput.

How the decision model estimate works

One question produces one typed decision. Monthly decisions equal requests × questions per request. Several questions can share a state, but their measured latency need not scale linearly.

For sequential requests, capacity = provisioned hours × 3,600 ÷ measured latency in seconds. Active inference hours = requests × latency ÷ 3,600. Utilization is active hours divided by provisioned hours. This is average capacity, without burst, queueing or failover headroom.

Dedicated compute is billed for the time it stays provisioned, including idle time. Cloud compute = full instance hourly price × provisioned hours. Local electricity = wall kW × hours × electricity price × PUE. Local monthly TCO adds labor and system cost ÷ depreciation months.

Jev API cost = requests × average billed input tokens ÷ 1,000,000 × $0.042. The token input affects this comparison only; it never adjusts self-host latency. Use actual usage.input_tokens and include retries in request volume. Jev also requires state + longest question to fit 32k tokens within its 64k total request limit.

Payback divides the upfront system cost by Jev API charges minus local operating costs, excluding depreciation from the cash savings. It is shown only when savings stay positive across the labor range and one endpoint has enough average capacity.

Self-hosting or a decision API?