How Much Does It Cost to Self-Host an LLM?
Pick an open-weight LLM such as Kimi, GLM, DeepSeek or Qwen, then compare the cost to run an LLM on owned hardware or rented GPUs.
Electricity for hardware you already own
This total includes only the electricity for the whole machine during the hours entered. Equipment purchase, depreciation, labor and cloud rental are excluded. It replaces the ownership estimate; do not add the two electricity totals.
Choose the currency on your electricity bill. Changing currency does not convert the rate; enter the new rate yourself.
Enter measured average power for the entire system, including CPU, RAM and PSU losses. GPU TDP is not a whole-machine measurement. No extra PUE multiplier is applied.
Enter elapsed running hours. Divide measured seconds by 3,600. Monthly time is limited to 744 hours; include idle hours only if the same average power covers them.
Use the variable energy rate on your bill. Fixed connection fees are separate.
Enter finite, non-negative power, hours and price. Maximum: 10,000,000 W, 1,000,000 task hours or 744 monthly hours, and 10,000 per kWh. Zero is allowed.
Time is entered directly here. Other workload calculators derive it only where matching performance data exists. Output duration, model memory fit and a GPU name do not establish generation time. For your own setup, measure the complete task, including offload and retries, before estimating its electricity.
Model and license details
- Repository
- deepseek-ai/DeepSeek-V4.1-Flash
- Variant
- GA (2026-09-10)
- Context window
- 1M (YaRN from 65,536)
- Weight format
- MXFP4 + MXFP8 Mixed (Safetensors)
- License
- MIT
- Verified
- 2026-09-15
License summary
DeepSeek-V4.1-Flash is published under MIT. The catalog records about 749B total parameters (552B backbone plus about 196B Engram), 8B active during prefill and 16B during decode, and a native MXFP4/MXFP8 mixed checkpoint. The official API model name is deepseek-flash; preserve the MIT notice and verify the exact checkpoint and bundled component terms before redistribution.
License summaries link to the model repositories. Check the license for the exact version you plan to use.
Cloud rental is unavailable for this model at this quantization. Its VRAM requirement is above the largest single instance (640 GB).
Advanced assumptionsDefaults are filled in; adjust only what you know.
Cost assumptions
Cost results and comparison
Edit inputsUpdating this configuration…
Found incorrect data? Report it to the administrator
How this estimate is calculated
The result shows the VRAM, hardware, electricity, labor, and source assumptions behind the estimate, so you can check what went into it.
VRAM Formula
VRAM ≈ params × bytes/param × runtime headroom (default ×1.2; adjustable for low/standard/high scenarios). FP16 = 2 bytes/param, INT4 = 0.5 bytes/param. For MoE models, the estimate uses total parameters to budget for all expert weights in GPU memory.
Hardware Matching
Prefer the model’s recommended hardware when it matches the selected precision, has enough VRAM and has a current price. Otherwise select the smallest priced tier with enough VRAM.
Electricity Calculation
Monthly electricity = power (kW) × running hours × rate × PUE. Power defaults to the single-GPU TDP reference; for multi-GPU or full-system rigs, enter measured wall power in the calculator. Default values are shown; adjust them to match your region and facility. Cloud rental already includes electricity.
Labor Estimate
We estimate monthly labor as a monthly salary range for an AI operations engineer × an FTE share. Default values are shown; adjust them to match your team and region. This is a planning estimate.
Data Sources & Updates
Model parameters, architecture, and model-card information come from official materials and Hugging Face; GPU specifications come from NVIDIA and other hardware vendors. Cloud and local-hardware prices use published public prices, channel or market references, or estimates on an entry-by-entry basis. Each price shows its update time, conditions, and source link when available; estimates without a verifiable link are marked as having no recorded source link.
The current public snapshot contains cloud prices from AWS, Google Cloud, and Microsoft Azure, plus manually checked local channel and market references. Model and hardware information is checked against official sources and Hugging Face. Use the provider, update time, and source link shown for each entry.
API workload budget
Estimate metered usage in USD from the stored tariff. Include every attempt and retry. Taxes, credits, free allowances and fixed fees are separate. This is independent of the self-hosting capacity estimate.
This tool covers uncached text requests up to 32,000 total tokens each, with input and output billed separately. Longer or cached requests are not estimated here. Select the applicable peak/off-peak rate.
Estimated usage cost: $0.27
Input: $0.15 · Output: $0.12 · $0.15 / $0.60 per 1M input / output tokens
Tariff source and verification date ·
global
Developer tools
LLM
Call hosted APIs and local model endpoints from a command line or Python application.
Local inference
vLLM
Serve a shared model API and plan GPU memory, concurrency and complete server costs.
Explore AI models
Compare model capabilities, API access and self-hosting costs. Choose a guide for your task.
Agent
Choose an agent for a specific task. Compare setup, model access and the cost of a complete run.
Palette
Generate editable slide decks with IBM’s designer, coder and repair workflow. Check model-backend access and complete deck costs before setup.
Read model guideagent-desktop
Connect an AI agent to native desktop apps. Check platform permissions, accessibility coverage and model-loop costs.
Read model guideDeepSeek Harness
Configure a model and workspace, create persistent plugins with Creator mode, and budget complete agent sessions.
Read model guideAura Autopilot
Set up an AI game tester for Unreal, Unity or Godot. Compare Aura, API keys and coding-agent subscriptions before budgeting a playtest.
Read model guidedsh-superagent
Explore DeepSeek’s research coding-agent prototype: repository tasks, deployment requirements, permissions and the cost of parallel work.
Read model guideStart with a budget for self-hosting an LLM
Turn the cost to run an LLM into a number your team can check, compare and use in a build-or-buy discussion.
Free model weights do not make a deployment free. The bill can include hardware, power, and the people who keep the system running. It also depends on the model, capacity, and deployment choice.
Use the estimate to decide whether self-hosting belongs in your AI plan, then walk your team through the budget.
The total is only the starting point. The result shows the cost lines and assumptions behind it, so you can explain where the budget goes.
What determines the cost to run an LLM?
The estimate starts with four things: the model, how you deploy it, how much you use it, and the work required to keep it running.
Model and size
Model size and precision affect the VRAM requirement and GPU cost for a self-hosted LLM.
Deployment option
Self-hosting an LLM on rented GPUs or purchased hardware leads to different upfront and monthly bills.
Expected usage
Monthly runtime sets the cloud rental and electricity costs. The presets cover 120, 360 and 720 hours per month.
Local operations cost
Someone still has to deploy, monitor, upgrade, and respond to incidents. The cost depends on your team and region.
You can adjust exchange rate, electricity rate, salary range, FTE share, PUE, system multiplier, and depreciation period.
Monthly runtime sets the cost estimate. Plan request capacity with throughput tests at your target context length and concurrency.
How the LLM cost calculator builds an estimate
For a self-hosted LLM, the calculator combines the model's hardware needs with your deployment choice, monthly runtime and operating assumptions. Each input affects the budget.
- Model and size data
- Parameter counts, architecture, and model sizes come from official documentation and Hugging Face.
- Price data
- GPU, hardware, and cloud prices are dated, manually checked references. Individual entries may come from official price pages, public product pages, channels, or third-party markets; the result shows each entry's update time, conditions, and source link when available.
- Cloud billing model
- Cloud prices use pay-as-you-go hourly rates. Reserved capacity, negotiated rates, and volume discounts are not included.
- Running time
- Light, medium, and heavy usage correspond to roughly 120, 360, and 720 hours per month.
- Operations cost
- Monthly labor uses a local monthly salary multiplied by an FTE share. It is not a fixed vendor fee.
- Exchange rate
- The default is 1 USD = 7.2 CNY. Change it in the calculator if your planning currency differs.
- Adjustable inputs
- Exchange rate, electricity rate, labor salary, FTE share, PUE, system cost multiplier, and depreciation period can all be adjusted to match your region and facility.
What this estimate leaves out
Allow for these additional production costs and capacity requirements when planning a self-hosted LLM.
- This is a planning estimate, not a vendor quote.
- Provider, region, contract, and availability can change the price.
- Real capacity also depends on throughput, concurrency, context length, quantization, and latency targets.
- Model quality is not part of the price estimate.
- Migration, fine-tuning, security, compliance, and application development can add cost.
- Production requirements such as redundancy and incident response can change the total.
How to budget for self-hosting an LLM
Choose a model, describe the setup, and leave with a shareable number for the budget conversation.
- 01
Choose a model
Pick the model and size closest to what you plan to run.
- 02
Describe the setup
Compare the cost to run an LLM in the cloud or on local hardware, then set runtime and cost assumptions for your region.
- 03
Check and share the estimate
Review the self-hosted LLM budget and each cost line, then share the URL with your team.
Should you self-host an LLM?
The calculator covers the self-hosting side. Your workload, budget, and team's ability to operate the system decide which option fits.
Self-hosting an LLM may fit better when:
- You need more control over your data and infrastructure.
- Your usage is stable enough to plan capacity.
- An open model meets your quality needs.
- Your team can deploy and maintain the system.
- Private deployment or customization has clear value.
A managed API may fit better when:
- You want to ship quickly.
- Usage is low, uncertain, or highly variable.
- Your team does not want to run GPU infrastructure.
- You need a proprietary frontier model.
- You would rather pay for what you use.
Both can be sensible. When a current public price is available, the calculator places the self-hosting estimate alongside the original model vendor's API reference price.
What the calculator gives you
Use How Much To Run AI as a starting point: see the one-time and monthly costs before deciding whether self-hosting deserves a closer look.
- Estimated one-time deployment cost range
- Estimated monthly operating cost
- Breakdown for compute, electricity, and operations labor
- The model and deployment assumptions used
- Adjustable assumptions: exchange rate, electricity rate, labor cost, PUE, system multiplier, and depreciation period
- Shareable link: your setup is stored in the URL, so others see the same result. You can also export the result as a PNG or PDF report.
Open models currently covered
Use the LLM cost calculator to compare the open model families and sizes covered here.
Kimi
Available sizes:
- Kimi K32.8TMoE
GLM
Available sizes:
- GLM-5.2753BMoE
- GLM-5.3753BMoE
- GLM-5.3-Flash320BMoE
DeepSeek
Available sizes:
- DeepSeek-V3671BMoE
- DeepSeek-V4-Flash-0731284BMoE
- DeepSeek-V4.1-Flash749BMoE
- DeepSeek-V4-Pro-08131.7TMoE
Qwen
Available sizes:
- Qwen 3.827B
- Qwen 3.8 Flash-Next180BMoE
Llama
Available sizes:
- Llama 3.18B / 70B
Gemma
Available sizes:
- Gemma 4E2B (5.1B total) / E4B (8B total) / 12B Unified / 26B / A4B / 31B DenseMoE
Frequently asked questions
These answers cover the questions that change the estimate from one setup to another.