What This AI Cost Calculator Includes

This page explains what the AI cost calculator estimates and how to read the model, memory, deployment, and cost terms in its results. See How it works for the defaults and formulas used in the estimate.

What the AI cost calculator estimates

Use this self-hosting calculator to budget for an open-weight LLM on local hardware or a priced cloud instance. Choose a model, precision, deployment path, usage preset, and exchange rate to get a planning estimate that may include:

• A one-time local hardware or machine cost range.
• A monthly breakdown for compute, electricity, operations labor, and a local hardware depreciation reference.
• Cloud rental cost when the current published pricing data contains a compatible tier.
• A comparison between local depreciation and cloud compute when both are available.

Try the LLM cost calculator to compare a model and deployment path.

Model size and MoE

Models are often described by parameter count. 7B means approximately 7 billion parameters; 14B and 70B describe larger parameter sets. More parameters usually mean a larger weight set and a higher memory requirement, which can move the recommendation to a different hardware tier.

Parameter count does not determine quality, speed, licensing, or fit for an application. The cost estimator uses total parameter count in its memory formula. For a Mixture-of-Experts (MoE) model, the estimate budgets for all expert weights in GPU memory and uses total parameters.

The catalog shows the model name, family, parameter size and whether it uses MoE. Where a repository is linked, you can find the available versions, weight formats and download instructions there. See the model license reference for license summaries and repository links.

Precision and quantization

Here, precision describes the approximate memory representation used by the cost estimator. The current options use these bytes-per-parameter assumptions:

• FP16: 2 bytes per parameter.
• INT8: 1 byte per parameter.
• INT4: 0.5 bytes per parameter.
• GGUF Q4: 0.55 bytes per parameter, as a simplified allowance for a Q4 GGUF variant.

The cost estimator uses these average weight sizes. Actual checkpoint size and runtime memory vary with the quantization format and inference framework. GPTQ, AWQ, FP8 and individual GGUF variants can have different memory requirements. Check actual checkpoint size and runtime memory when choosing a hardware tier.

VRAM and hardware fit

VRAM is the fast memory available on a GPU or accelerator. The self-hosting calculator uses a simplified estimate of weight residency and basic runtime reserve. With the default memory overhead, the formula is:

required VRAM ≈ total parameters (B) × bytes per parameter × 1.2

Change the default 1.2 reserve with the memory-overhead scenario in the calculator. Actual deployment memory can also include KV cache, activations, framework workspace, unquantized layers, replication or sharding overhead, and safety margin. Long context, high concurrency, and special architectures can make the estimate low or conservative.

The self-hosting calculator first considers the model’s recommended hardware when it fits the memory estimate and has a current price. Otherwise it selects the smallest priced tier with enough VRAM. Cloud matching covers up to 640 GB of combined GPU memory on a single priced instance.

For a multi-GPU deployment, your inference software needs to split the model across the selected GPUs, with a suitable interconnect between them.

Deployment options

Cloud rental uses a compatible published pay-as-you-go price and scales it by the selected monthly runtime. Cloud electricity is included in the rental price. Operations labor is added using your selected salary and FTE share.

Local purchase uses the published local hardware price range as a budget reference. The AI cost calculator applies your system multiplier, then adds recurring electricity and operations costs and shows a monthly depreciation reference. The default multiplier is 1.4. Adjust the multiplier for the CPU, RAM, chassis and other components in your setup.

The cheaper option depends on utilization, region, inventory, account terms, discounts, interconnect, support, electricity, and the team's operating requirements.

Usage intensity

The LLM cost calculator uses a monthly runtime preset to scale recurring costs. It describes billed or powered-on hours. Work out request capacity separately from measured throughput, latency and concurrency. The exact presets and formulas are listed in How it works.

What is included

Depending on the deployment path, the AI cost calculator may include:

• Published local hardware or cloud compute references with recorded source and update information.
• Local electricity using GPU TDP or your measured wall power, runtime, electricity rate and PUE.
• Operations labor using your salary range and fraction of a full-time equivalent (FTE).
• A local hardware depreciation reference over your selected period.
• Currency conversion between USD and CNY using the selected exchange rate.
• Price details: provider, hardware or instance name, region, currency, billing unit, update date, notes and a source link when available.

What is not included

Budget separately for application development, migration, fine-tuning, evaluation, security, backups and redundancy. Provider extras such as tax, storage, network transfer and support depend on the listing.

Context length, KV cache, concurrency, batching and the inference framework affect memory and throughput. Measure these with the workload you plan to serve.

The AI cost calculator shows price notes, update dates and source links where available. If a price is a reference estimate or has no recorded source link, that is shown beside it. Model download requirements and permitted uses are described in the repository and model license reference.

Adjusting the default assumptions

The self-hosting calculator starts with defaults for memory overhead, electricity, PUE, labor, depreciation, full-system cost and exchange rate. Adjust these to match your setup. Precision determines the bytes-per-parameter assumption. Measured memory and power use help you choose values for your workload. The exact values, formulas, and worked examples are on How it works, so you can see how each input affects the estimate.

FAQ

Why might the AI cost calculator have no cloud result for a model?

The AI cost calculator needs a current price for a hardware tier that fits the memory estimate. If there is no priced match, or the requirement exceeds the 640 GB cloud matching range, the result shows as unavailable.

Is aggregate VRAM the same as one large GPU?

No. Combined VRAM is spread across separate GPUs. Your inference software needs to split the model across them, and the interconnect affects performance.

Does 720 hours mean the GPU is fully utilized?

No. It means the system runs for 720 hours in the month, whether it is busy or idle. Request capacity depends on your measured throughput and workload.

Why can the electricity line differ from my power bill?

The cost estimator uses GPU TDP unless you enter measured whole-system wall power, then applies your electricity rate and PUE. The TDP fallback excludes host components such as the CPU, RAM and fans. A wall-power measurement gives a closer reference for the full machine at that workload.

Does a lower estimate mean a better deployment?

A lower estimate can reflect less running time, different hardware or a smaller staffing allowance. Compare those assumptions alongside the speed, reliability and support your workload needs.

Quick glossary

• Parameters: Learned values in a model. Their count is a practical input for this simplified memory estimate.
• Quantization: Representing parameters with fewer bits to reduce memory use, usually with a quality or compatibility trade-off.
• VRAM: GPU or accelerator memory used to load model parameters and runtime state.
• MoE: Mixture of Experts; the current calculator uses total parameters for its VRAM estimate even when active parameters are lower.
• Deployment: The infrastructure path used to serve or run the model, such as cloud rental or local hardware.
• Usage preset: A monthly runtime assumption used to scale recurring cost estimates.
• Pay-as-you-go: Cloud billing based on the time or usage of an instance rather than an upfront purchase.
• FTE: Full-time equivalent, used here to express the estimated share of an operations role.

Choose a model cost calculator

Use the calculator for your workload: LLM running costs, image generation costs, video generation costs, TTS self-hosting costs, or decision model costs. Browse the model guides for capabilities, access and model-specific cost details.

AirLLM loads weights a layer at a time, trading GPU residency for disk transfers. Read the AirLLM setup and running-cost guide before using this calculator for a streaming deployment.