LLM CLI: call cloud and local models from one client

LLM is Simon Willison’s open-source command-line tool and Python library for working with language models. Use the same client to send a prompt, attach context, save a reusable template or inspect a conversation across supported providers. Its plugins and custom endpoints connect the client to models; a local inference server still runs the model itself.

Choose LLM when you want model calls in a shell script, a repeatable comparison or a small Python application. A hosted endpoint bills you for its requests. A local endpoint consumes your machine or rented server time. The client code is Apache-2.0 licensed, with no LLM subscription required.

Install LLM and choose a model explicitly

The 0.37 release adds GPT-6.1 Sol and Terra and an easier command for saving OpenAI-compatible endpoints. Use 0.37.1 for a new installation: its fixes cover malformed endpoint YAML, omitted system prompts in certain Python API calls and non-streaming servers that omit usage information.

uv tool install 'llm==0.37.1'
llm --version
llm models list
llm models default

For an existing uv installation, use uv tool upgrade llm. Install plugins into LLM’s own environment with llm install, rather than a different Python environment.

The default model is gpt-6-luna when you have not configured your own default. Save a chosen default with llm models default MODEL_ID, or pass -m MODEL_ID in a script so a later default change does not change its backend. For direct OpenAI requests, llm keys set openai prompts for your key without putting its value in the command. API billing is separate from a chat subscription.

Compare the existing GPT-6 Luna, GPT-6.1 Sol and Terra guides for model-specific access and pricing. Installing the client does not give an account access to every model it lists.

Connect LLM to a local or compatible endpoint

First start a supported model server. The vLLM setup guide shows a local endpoint serving Qwen/Qwen3.8-27B under the name local-qwen. Register that same served name in LLM:

llm openai endpoint add http://127.0.0.1:8000/v1 \
  -m local-qwen --id local-vllm
llm -m local-vllm 'Write a three-item checklist for reviewing a bug fix.'

The API base must include the path expected by your server. -m in the endpoint command is the server’s model name; --id is the local name you subsequently pass to LLM. Confirm the server’s advertised models if a request returns “model not found.”

For a remote endpoint requiring authentication, save a named key with llm keys set NAME, then add --key NAME when registering it. LLM records the key’s name in extra-openai-models.yaml, rather than copying the secret into that configuration. It does not automatically send your saved OpenAI key to a different API base.

Declare endpoint capabilities such as --supports-tools, --supports-schema or --vision only when the exact server and model support them. A compatible request format does not make all providers implement the same tools, attachments, structured output or accounting. Test one plain text request before adding those capabilities.

Keep useful logs without confusing them with a bill

LLM logs prompts and responses to a local SQLite database by default. Inspect configuration with llm logs status; use llm logs off to disable default logging or --no-log for an individual prompt. Saved keys and logs are local files, so include them in your workstation’s access and backup decisions. Disabling local logging does not change the remote provider’s own data policy.

Logs help compare prompts, models and retries. Token information depends on what the backend returns. A successful response without usage fields is not evidence of a free request; reconcile a hosted experiment with the provider’s usage records.

The client also supports templates, fragments, embeddings and tool plugins. Add tools when the task requires them, with permissions appropriate to the files or commands they can access. For an autonomous repository workflow, compare DeepSeek Harness, which supplies an agent loop beyond a model-call client.

Budget a complete task

For hosted calls, add fresh input, cached input and output at the route’s applicable rates across every attempt. As a calculation example, 20,000 input tokens and 2,000 output tokens at assumed rates of $1 and $4 per million cost $0.028. Three identical attempts cost $0.084. These are example rates, not a quote for a particular provider.

For local calls, count model loading, prompt processing, generation and idle server time. A $2/hour server kept running for 45 minutes costs $1.50 even if a prompt itself finishes quickly. Software installation and troubleshooting take time as well. Use the text calculator and budget notes to compare a supported self-hosted model with the actual hosted route.

Start with one model and one saved prompt. Check the answer, the selected backend and its usage before turning that call into a larger batch.

Sources

Checked 2026-11-04: LLM releases and patch fixes, installation, keys and defaults, custom endpoints, logging and source and license.