AI Deployment Cost Calculator | How Much To Run AI

This page explains what the calculator estimates and how to read the model, memory, deployment, and cost terms in its results. See How it works for the exact constants and formulas used by the current implementation.

What this calculator estimates

How Much To Run AI estimates the cost of running the model records in its current open-weight catalog on local hardware or a priced cloud instance. Choose a model, precision, deployment path, usage preset, and exchange rate to get a planning estimate that may include:

• A one-time local hardware or machine cost range.
• A monthly breakdown for compute, electricity, operations labor, and a local hardware depreciation reference.
• Cloud rental cost when the current published pricing data contains a compatible tier.
• A comparison between local depreciation and cloud compute when both are available.

This is a planning estimate. It is not a cloud provider quote, hardware quotation, capacity plan, or performance guarantee. Start with the calculator when you are ready to compare a model and deployment path.

Model size, MoE, and model identity

Models are often described by parameter count. 7B means approximately 7 billion parameters; 14B and 70B describe larger parameter sets. More parameters usually mean a larger weight set and a higher memory requirement, which can move the recommendation to a different hardware tier.

Parameter count does not determine quality, speed, licensing, or fit for an application. The estimator uses total parameter count in its memory formula. For a Mixture-of-Experts (MoE) model, active parameters are informative metadata only. The calculator does not substitute them into the VRAM calculation because the current logic assumes all expert weights must be resident.

The catalog contains a model name, series, displayed size, and MoE flag. Seeded records may also include a Hugging Face model ID. The calculator does not yet present a complete version, context-window, weight-format, or license record for each checkpoint. See the separate model license reference for the current catalog's license summaries. Use model labels as inputs to the cost model, not as legal or compatibility conclusions.

Precision and quantization

Here, precision describes the approximate memory representation used by the estimator. The current options use these bytes-per-parameter assumptions:

• FP16: 2 bytes per parameter.
• INT8: 1 byte per parameter.
• INT4: 0.5 bytes per parameter.
• GGUF Q4: 0.55 bytes per parameter, as a simplified allowance for a Q4 GGUF variant.

These values are planning approximations, not promises about a particular checkpoint file. GPTQ, AWQ, FP8, and different GGUF variants are not modeled separately today. Quantization can reduce memory use, but quality, compatibility, unquantized layers, runtime overhead, and actual file size still depend on the model and inference stack.

VRAM and hardware fit

VRAM is the fast memory available on a GPU or accelerator. The current formula is a simplified estimate of weight residency and basic runtime reserve:

required VRAM ≈ total parameters (B) × bytes per parameter × 1.2

The fixed 1.2 reserve is not a context-length or concurrency model. Actual deployment memory can also include KV cache, activations, framework workspace, unquantized layers, replication or sharding overhead, and safety margin. Long context, high concurrency, and special architectures can make the estimate low or conservative.

The matcher first considers a recommended model-to-hardware mapping when it is still priced and large enough. Otherwise it selects the smallest currently priced hardware tier with enough recorded VRAM. For cloud matching, the current availability check is capped at 640 GB of aggregate VRAM on a single priced instance. This is a limit of the current catalog and matching data, not a claim that larger bare-metal, multi-node, or other cloud deployments do not exist.

Aggregate GPU memory is not automatically one shared memory pool. A multi-GPU or multi-node deployment still needs compatible tensor or expert parallelism, sharding, and interconnect support. The current estimator does not validate those conditions.

Deployment options

Cloud rental uses a compatible published pay-as-you-go price and scales it by the selected monthly runtime. Cloud electricity is treated as included in the rental price. The current result still includes the operations-labor assumption.

Local purchase uses the published local hardware price range as a budget reference. The calculator applies a fixed full-system multiplier, then adds recurring electricity and operations costs and shows a monthly depreciation reference. The multiplier is a fallback planning assumption, not a bill of materials or a confirmed workstation or server quote.

The cheaper option depends on utilization, region, inventory, account terms, discounts, interconnect, support, electricity, and the team's operating requirements.

Usage intensity

Usage intensity is a monthly runtime preset that scales recurring costs. It controls billed or powered-on hours in the cost model. It does not infer requests per second, token volume, throughput, latency, concurrency, or GPU utilization. The exact presets and formulas are listed in How it works.

What is included

Depending on the deployment path, the estimate may include:

• Published local hardware or cloud compute references that pass the current publication and freshness boundary.
• Local electricity using GPU TDP, runtime, a reference electricity rate, and the current PUE simplification.
• Operations labor using a fixed salary range and fraction of a full-time equivalent (FTE).
• A local hardware depreciation reference over a fixed period.
• Currency conversion between USD and CNY using the selected exchange rate.
• Price provenance shown with the result: source/provider, resource identifier, region, currency, billing unit, update time, condition, and source link when available.

What is not included

The result does not fully price application development, migration, fine-tuning, evaluation, security, compliance, backups, high availability, monitoring, commercial support, taxes, or negotiated contracts. It also does not model exact context length, KV cache, concurrency, latency, throughput, batch size, driver or framework compatibility, or production load.

The calculator result does not yet include a complete license summary or gated-access status for each checkpoint. Use the separate model license reference as a starting point, then verify the exact repository. The UI also does not classify price records as official direct price, calculator/API estimate, marketplace, retail, linear estimate, or quote-request price. Review the source conditions and verify the model checkpoint before making a purchase or deployment decision.

Current assumptions are estimates

The calculator uses fixed planning assumptions for runtime reserve, bytes per parameter, electricity, PUE, labor, depreciation, full-system cost, and exchange rate. These assumptions create a comparable scenario; they are not measurements from a particular facility or workload. The exact values, formulas, and worked examples are on How it works, where you can check them against the implementation.

FAQ

Why might a model have no cloud result even though it can run somewhere?

The current result uses active, published, non-expired price observations and the supported priced hardware tiers. A missing cloud result means that dataset and matching path have no current match. It does not prove that no optimized, multi-device, bare-metal, or other provider option exists.

Is aggregate VRAM the same as one large GPU?

No. The current match compares recorded aggregate capacity. A real deployment must also support model sharding and the required interconnect. Capacity alone does not prove that the model will load or perform well.

Does 720 hours mean the GPU is fully utilized?

No. It is the heavy monthly runtime preset used to scale the current cost model. It does not represent traffic, tokens, utilization, or a capacity target.

Why can the electricity line differ from my power bill?

The current calculation uses GPU TDP as a proxy and applies a reference electricity rate and PUE. It does not measure wall power or fully model the CPU, memory, storage, fans, network, PSU efficiency, average load, or facility contract.

Does a lower estimate mean a better deployment?

No. A lower number may reflect different hardware, availability, performance assumptions, support, reliability, or license conditions. Use the estimate to frame a budget conversation, then validate the intended checkpoint and workload.

Quick glossary

• Parameters: Learned values in a model. Their count is a practical input for this simplified memory estimate.
• Quantization: Representing parameters with fewer bits to reduce memory use, usually with a quality or compatibility trade-off.
• VRAM: GPU or accelerator memory used to load model parameters and runtime state.
• MoE: Mixture of Experts; the current calculator uses total parameters for its VRAM estimate even when active parameters are lower.
• Deployment: The infrastructure path used to serve or run the model, such as cloud rental or local hardware.
• Usage preset: A monthly runtime assumption used to scale recurring cost estimates.
• Pay-as-you-go: Cloud billing based on the time or usage of an instance rather than an upfront purchase.
• FTE: Full-time equivalent, used here to express the estimated share of an operations role.