How the Self-Hosted AI Cost Calculator Works

The AI cost calculator combines your model, deployment choice and operating assumptions into a planning estimate. The current flow is:

model + precision + deployment + usage + exchange rate
                         ↓
       VRAM estimate + published-hardware match
                         ↓
          listed prices + runtime assumptions
                         ↓
       upfront cost + monthly breakdown + conclusion

More info explains which costs are included and how to adjust the assumptions.

Step 1: Choose a model and precision

The LLM cost calculator uses the selected model's parameter count and precision to estimate memory. Precision determines the bytes per parameter:

For MoE models, the estimate budgets for all expert weights in GPU memory and uses total parameters.

Step 2: Estimate VRAM and match hardware

The cost estimator calculates required VRAM with:

required VRAM (GB) = parameters (B) × bytes per parameter × memory overhead

The AI cost calculator defaults to a 1.2 memory-overhead factor for weight residency and basic runtime reserve. Change the memory-overhead scenario to use a different factor.

Here are a few examples using the default 1.2 factor:

7B FP16:  7 × 2 × 1.2   = 16.8 GB
7B INT4:  7 × 0.5 × 1.2 =  4.2 GB
32B FP16: 32 × 2 × 1.2  = 76.8 GB
32B INT4: 32 × 0.5 × 1.2 = 19.2 GB

Actual memory also depends on context length, concurrency, KV cache, activations, unquantized layers and the inference framework. Replication and sharding can add overhead. Test your intended workload to size the remaining headroom.

The self-hosting calculator matches hardware in this order:

1. Prefer the model’s recommended hardware when it fits your selected precision and has a current price.
2. Otherwise choose the smallest priced hardware tier with enough VRAM.
3. Cloud matching covers priced instances with up to 640 GB of combined GPU memory. Larger requirements show as unavailable.

For multi-GPU setups, your inference software needs to split the model across the selected GPUs. The interconnect also affects how well the GPUs can work together.

Step 3: Apply the deployment path and price data

Cloud

For a cloud deployment, the cost estimator uses published prices for the matched hardware and calculates:

cloud monthly compute = hourly price × selected monthly hours

The range spans the lowest and highest listed prices after currency conversion. Cloud electricity is treated as part of the rental price. The monthly result also includes labor based on your salary range and FTE share.

Local

For a local deployment, the AI cost calculator applies your system multiplier to the published hardware price range:

one-time local system estimate = hardware price × system multiplier

The default 1.4 multiplier adds a budget allowance for the chassis, CPU, RAM, PSU and other components around the GPU. Adjust it to match your setup. Some multi-GPU prices are calculated from a single-GPU reference; their price notes explain the basis.

Step 4: Apply usage and operating assumptions

The self-hosting calculator uses these monthly runtime presets:

These hours describe how long the system runs each month. To work out how many requests it can serve, measure throughput and latency with your workload.

For local deployment, the cost estimator calculates electricity with:

power (kW) × monthly hours × electricity rate × PUE

The defaults are 0.65 CNY/kWh and PUE 1.2; adjust them to match your electricity contract and facility. Power uses GPU TDP unless you enter measured whole-system wall power. The TDP fallback excludes CPU, RAM, storage, fans and other host components. The displayed electricity range is approximately 90%–110% of the result.

The AI cost calculator estimates operations labor as monthly salary × FTE share. At the default values:

15,000–25,000 CNY/month × 0.15 FTE
= 2,250–3,750 CNY/month

For local deployment, the monthly hardware reference divides the maximum one-time estimate by your selected depreciation period, which defaults to 36 months. Use it to compare hardware purchase with cloud rental. The default exchange rate is 1 USD = 7.2 CNY; you can change it in the calculator.

Step 5: Read the result

Depending on deployment and available prices, the AI cost calculator can show:

• Estimated VRAM and matched hardware.
• One-time local deployment cost.
• Monthly compute, electricity, operations labor, and total cost.
• Local depreciation reference and, where possible, a cloud comparison.
• Price details: provider, hardware or instance name, region, currency, billing unit, update date, notes and a source link when available.
• A conclusion for the selected scenario.

Monthly costs are shown in USD and CNY using your selected exchange rate. When there is no current price for a hardware tier, the result shows it as unavailable.

Where the prices come from

The cost estimator shows how each price was obtained and whether it is a reference estimate:

A price marked as an estimate may be a market reference or a calculation from a smaller system. For multi-GPU systems, check the listed GPU count and which host components the price includes.

The price notes and source link show what each listing covers.

Comparing listed prices

The self-hosting calculator uses current listed prices. Each result includes the provider, region, billing unit, update date and price notes, with a source link when available.

Tax, storage, network transfer and support may be included or billed separately by the provider. Check the price notes when comparing providers. Reserved, spot and negotiated rates can differ from the listed rates.

Adjust the estimate to your setup

The LLM cost calculator uses these formulas with your selected inputs. Replace the defaults with your electricity rate, staffing costs and measured power use. See More info for the costs covered by the estimate.

Choose a model cost calculator

Use the calculator for your workload: LLM running costs, image generation costs, video generation costs, TTS self-hosting costs, or decision model costs. Browse the model guides for capabilities, access and model-specific cost details.