Edge0-145B and 120B: Local Setup, Memory & Power Costs

The two checkpoints are derived from Qwen3.8-Flash-Next and route each token to 10 of 512 experts, plus one shared expert. The 145B model has 10.12B active parameters per token; the 120B model has 8.64B. Each supports a 256K context in the model profile, with an int8 KV cache and 32K tokens resident in the published configuration.

Each model is available in a Q3_K folder and an NVFP4 folder. The complete folder sizes are larger than the expert weights alone because each also contains shared layers, routing data, embeddings, a tokenizer and a speculative decoding draft. Leave free disk space for the whole download and use the folder as supplied; edge0-infer reads its model identifier from model.json.

The latest 145B and 120B release targets Linux or Windows with an NVIDIA GPU and CUDA 13. The runtime guide lists at least 16 GiB of VRAM and 20 GiB of system memory. Its published low-memory measurements use an RTX 5090 D constrained to that budget, covering 145B Q3_K and both 120B formats. The profile provides no corresponding 16/20 result for 145B NVFP4. Format, context and expert placement affect the complete allocation; leave room for the operating system and client as well as the model process.

The model weights exceed the VRAM in the tested 24 GB RTX 5090 D setup, so the engine keeps selected experts on the GPU and reads others from host memory. In the authors’ residency report, the 145B Q3_K weights occupied 23.14 GiB of VRAM plus 31.28 GiB of RAM; 145B NVFP4 used 23.25 plus 38.27 GiB. The 120B Q3_K figures were 22.93 plus 18.40 GiB, and NVFP4 23.02 plus 23.40 GiB. These are reported weight residency totals, not extra space requirements to add to the full download; KV cache, runtime buffers, the OS and other applications still matter.

The 24 GB RTX 5090 D headline measurements are developer-reported: Q3_K agent decode was 121.2 tokens/s for 145B and 144.6 for 120B. Prefill was 3554.9 and 4576.5 tokens/s respectively at a 16K-token input. The repository lists additional cards and builds, but these figures are not independent measurements. Its Terminal-Bench 2.1 results are also model-level results rather than separate quality runs for Q3_K and NVFP4.

Build both pieces from the Edge0 repository: edge0-strata is the CUDA 13 engine for these checkpoints, and edge0-infer provides the command line and local API. After starting serve, use the loopback API address printed in the terminal, then send a short chat through an OpenAI-compatible client. For a terminal conversation, use chat instead of serve with the same model folder. Start with a bounded context, such as --context 32768; the model profile’s 256K capacity is distinct from the CLI default of 131072. The repository documents separate Linux and Windows build steps. Download one model folder in one format, then pass the folder containing model.json to the runtime. Do not point the command at the parent directory holding multiple variants.

On Linux, install CUDA 13, CMake 3.24 or newer, Ninja, a C++17 compiler, Python 3.10 or newer, uv and Git before following the pinned build below. The CPU preset builds the wrapper; inference still uses the separately built NVIDIA engine. The Hugging Face CLI downloads the complete public folder. For example, choose one of these repository IDs: Edge0/Edge0-145B-q3k, Edge0/Edge0-145B-NVFP4, Edge0/Edge0-120B-q3k, or Edge0/Edge0-120B-NVFP4. Downloading a second format requires disk space for that full folder too.

Choose 120B for the first trial when host RAM or responsiveness is the constraint. In the published 24GB RTX 5090 D Q3_K setup, 120B uses 18.40 GiB of host RAM for weight residency against 31.28 GiB for 145B, with similar GPU residency. The authors report 144.6 versus 121.2 decode tokens per second. Leave additional memory for context, buffers and the operating system.

Try 145B when the workstation has the extra memory and a representative task shows a useful quality gain. The speed and residency results above do not establish that one checkpoint completes your agent task more reliably. Keep the quantization, context length, tool environment and acceptance test fixed, then compare successful runs, total time and energy.

The project does not list a hosted API or per-token rate for these weights. Local inference avoids a provider token bill, while shifting cost to the computer, its electricity and maintenance. Measure power at the wall during the workload you care about: the GPU model alone does not tell you the whole system draw.

For example, assume a meter reads an average 350 watts while an agent is running for 40 hours in a month. That is 0.35 kW × 40 h = 14 kWh. At an assumed electricity rate of $0.20 per kWh, the energy charge is $2.80 for that month. Substitute your own wall-power reading, hours and utility rate; then add a share of the workstation purchase price, storage and any cooling costs if you are comparing against a hosted service.

A useful comparison records the task result as well as energy: accepted changes per run, failed attempts, first-token time, full completion time and the hardware configuration. A fast decode rate alone cannot show whether a local agent finished the task or how much the full system cost to keep available.

The model weights are not covered by the framework’s Apache-2.0 code license. Edge0-specific contributions use the Edge0 Community Model License 1.0, while Qwen materials remain subject to the Qwen Community License 1.0. The model card says the Edge0 license permits personal use, research, modification, noncommercial sharing, internal use and commercial use of generated outputs, subject to the upstream Qwen conditions.

Selling or commercially bundling the model, providing a commercial model service, or offering an independent commercial coding or office assistant using it requires separate written permission from Edge0. The model card also says Edge0 permission does not replace Qwen’s authorization or attribution requirements. Review both licenses and the repository NOTICE for your use case; the Edge0 framework code is Apache-2.0 and its Strata engine is MIT-licensed.

In a November 13 Reddit thread, LightBrightLeftRight describes trying the 145B Q3_K folder against Qwen Flash Next IQ3_S on a “3090 setup.” The report does not include the runtime revision, complete command or prompts. It raises a useful comparison question, but does not establish which model will finish a coding task faster on your machine. Keep the model folder, engine revision, input length and complete task result together when comparing runs.

For hardware support, an open GitHub issue asks about AMD Vulkan or ROCm/HIP support for 145B and 120B. A maintainer replied that an AMD-support pull request was under review on November 13. The published setup still targets NVIDIA and CUDA 13; check the current repository before buying or configuring hardware.