Llama-modchip: run DeepSeek-V4-Flash-0731 on a 64GB CPU machine

Llama-modchip is a set of llama.cpp patches, custom quantization tools and launch utilities for local MoE inference. Its maintainer reports fitting the 284B DeepSeek-V4-Flash-0731 checkpoint into a 64GB Linux machine without a GPU or swap. The converted weights occupy 56.56 GiB, with reported process memory around 57.6–58.9 GiB at a configured 256K context.

That is a tight memory budget. Start on a machine you already own, close other memory-heavy applications and use the documented patched runtime. The 64GB result uses an aggressive quantization of the exact 0731 checkpoint; it does not establish the same capacity for newer DeepSeek versions or the same speed on every CPU.

What llama-modchip changes

The project reduces an Unsloth UD-IQ1_S conversion from 76.87 GiB to 56.56 GiB using a mixed tensor allocation, about 1.71 bits per weight overall. Some shared and sensitive tensors retain higher precision while most routed expert weights use very low-bit formats. This is not a uniform two-bit conversion.

The runtime also changes CPU kernels and cache use, with optional speculative drafting. The converted GGUF uses a nonstandard tensor layout and needs its matching runtime. A stock llama.cpp, Ollama or LM Studio binary can fail while reading its tensors. Use the project's patched launcher or matching bundled binary and model.

The target is DeepSeek-V4-Flash-0731, with 284.33B total and 13.17B active parameters according to the project. DeepSeek-V4.1-Flash-1017 is a different checkpoint with different deployment requirements. Keep that distinction when comparing file sizes or copying a command.

Can it fit your 64GB machine?

The headline run is the maintainer's Ryzen 9 9955HX system: 16 cores, 32 threads, two CCDs, 64MB L3 and 64GB DDR5 on Linux. Its reported 57.6–58.9 GiB process footprint leaves little room for the operating system and other workloads. Check actual usable memory rather than comparing the weight file with the RAM label.

The project lists other CPU architectures and smaller MoE checkpoints, but the Ryzen DeepSeek result should not be transferred to a Mac or another processor. For a different machine, use its supported build path and measure startup, memory, prompt processing and generation separately.

Install the matching model and server

The project README provides an automated ./modchip chat path and a direct Linux x86_64 server path. The latter makes the exact model and binary explicit. Install the Hugging Face CLI first, then download the complete converted model repository:

hf download Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF \
  --local-dir ./dsv4
chmod +x ./dsv4/llama-server-modchip-linux-x86_64
./dsv4/llama-server-modchip-linux-x86_64 \
  -m ./dsv4/DeepSeek-V4-Flash-0731-Fractal-00001-of-00003.gguf \
  -c 32768 -t 16 -fa on --port 8080

Keep all three shards in the same directory. This example follows the project's 32K-context launch settings; the thread count comes from its example and should be tuned for your hardware. Check the terminal's loading messages and the local server before attaching a coding client. If loading fails with a tensor-shape or layout error, check the exact weight revision and matching runtime before changing memory settings.

The converted-weight repository identifies the base checkpoint and marks the weights MIT. The runtime repository is Apache-2.0. Keep the applicable notices when redistributing either component.

Read the speed claims by mode

The maintainer's detailed Zen 5 results separate cold prompt processing from cached agent turns. A 580-token cold prompt reached 43.98 tokens/s; the 44.36 tokens/s result is an agent turn with prompt caching, 403 total KV tokens and 339 new tokens. These short-prompt measurements do not establish the processing speed of a full 256K prompt.

Generation results also depend on the run mode: roughly 9–10 tokens/s for default Top-6 runs, with 11.5–14.7 tokens/s reported for speculative drafting. Draft acceptance, prompt content and configuration affect the latter range. Compare prompt processing, generation and the complete task separately, using the exact benchmark command and workload.

By default, the project evaluates all six routed experts plus the shared expert. Its “lossless” description concerns matching the patched and unpatched runtime on the same quantized checkpoint. It does not mean that 1.71-bit weights reproduce BF16 quality. Optional Top-5 or Top-4 expert pruning changes model computation. The maintainer reports speed gains and output drift that vary with the other enabled optimizations; leave pruning off for an initial comparison.

Read the DeepSeek optimization notes alongside the benchmark command, including context, thread settings, drafting and expert routing.

What other users have tested

The project's Threadripper benchmark discussion includes contributor testing on a Threadripper PRO 7985WX and a thread-count sweep. Its roughly 111.6 tokens/s warm speculative result concerns a 30B Nemotron checkpoint, not the 284B DeepSeek configuration. That distinction matters when reading the project leaderboard.

In the November 8–9 HN report by ar-nelson, a 24-core Haswell-era Xeon machine with 96GB DDR4 passed the project's readiness check but could not load the DeepSeek conversion through modchip start: it reported a tensor-shape mismatch. The bundled binary then failed with Illegal instruction. The user traced the binary incompatibility to the CPU's lack of AVX-512 and did not complete the run. A readiness grade alone therefore does not establish compatibility with the downloaded executable.

In a separate report by aappleby, a Ryzen 7945HX with 96GB DDR5 ran Nemotron with faster prompt processing but similar generation speed to plain llama.cpp. The commenter had not tested DeepSeek. This comparison adds a useful hardware-specific check, but it supplies no independent 284B memory or quality measurement.

For a first trial on an older CPU, confirm the exact executable's instruction-set requirements and available build instructions before downloading the large weights. Test representative prompts with known answers and a small repository task with an external test command, then compare accepted results with a higher-precision checkpoint or your current hosted workflow.

Budget the full machine session

CPU-only operation removes a GPU requirement, but the full cost still includes the computer, RAM, storage, electricity and time spent waiting or retrying. Measure wall power if possible; the processor's advertised power limit is not the machine's actual consumption.

For a planning example, a computer averaging 150 watts for four hours uses 0.6 kWh. At an assumed $0.20/kWh, electricity is $0.12, before hardware allocation or setup time. A rented CPU server adds its hourly charge for all reserved time, including downloads and failed runs. Divide that total by accepted completed tasks, not just generated tokens.

Use the local inference directory to compare general runtimes, Strata for a different PC MoE deployment approach, and the cost methodology for hardware and utilization assumptions. For the newer hosted or self-hosted model, use the separate DeepSeek V4.1 Flash guide.

Sources

Sources checked November 9, 2026: llama-modchip README, v2.4.0 release, converted weights and file list, optimization notes, and the linked community discussions. Performance figures above are attributed reports; the power examples are calculations from stated assumptions.