Clover 35B A3B: Diffusion, Local Setup and Memory Costs

A conventional language model extends a response one token at a time. Clover begins a block with masked positions, resolves confident tokens in parallel, then adds the completed block to the preceding context. Its hybrid Gated DeltaNet and attention backbone incorporates information from both directions within the current block while preserving the earlier causal prefix.

The Hugging Face implementation defaults to blocks of 32 tokens and a confidence threshold of 0.95. The SGLang and llama.cpp integrations also support self-speculative decoding: Clover proposes a block and uses its own causal path to verify what to accept. That path does not need a separate draft model. Acceptance rules, block size and runtime kernels affect the amount of useful work done per step.

Clover is text-only. The publisher describes a future decision-model release separately; that announcement does not make this checkpoint a typed-decision endpoint. For fixed-choice probabilities, use an available decision model such as Clef.

The BF16 quick start loads roughly 69 GB of weights on one CUDA GPU, with additional space needed for inference state. Approximately 3B active parameters describe per-token computation; the remaining experts still have to be stored. Longer prompts, larger blocks and concurrency add their own memory demands.

DiffuSpace now publishes three official GGUF quantizations. The smallest listed file is already larger than a 16GB GPU’s memory, so it cannot fit entirely in that VRAM before runtime overhead. A configuration that uses host RAM or offloading has a different capacity and speed profile from a fully GPU-resident run.

The project’s Python path uses PyTorch 2.10 or newer, Transformers 5.19.0 and a CUDA GPU with room for BF16 weights and state. Install PyTorch for your CUDA setup, then install the pinned Transformers version, accelerate and safetensors. The model loads custom repository code through trust_remote_code; inspect that code in an environment you control.

This shortened example caps generation at 512 new tokens for a first smoke test. Inspect whether the output answers the question before using a larger reasoning budget. Save the model revision, generation settings, peak memory and full elapsed time so later changes remain comparable.

As checked October 30, the linked SGLang and llama.cpp support pull requests remain open. Their integration code is available to test, but a generic installed release should not be assumed to contain it. The official GGUF card explicitly fetches llama.cpp pull request 31306 before building llama-speculative-simple.

Follow that card’s CUDA or CPU build and download one matching GGUF. Its sample uses draft-self with a raw, explicitly formatted chat prompt. The example’s prompt must fit the physical batch size; the runtime context and generated-token limit also need room for the thinking phase. Keep the acceptance flags with that example executable instead of transferring them to another frontend.

SGLang’s contributor describes separate threshold and exact acceptance modes and documents launch restrictions. Its open discussion includes a KV-slot reservation concern at full context. Begin with short prompts and bounded concurrency, then test the longer workload before exposing a service. The model’s 128K specification does not establish the capacity of an unfinished runtime integration.

DiffuSpace reports 680 tokens/s on a single GB300 using BF16 at batch size one on internal agent workloads, and 515 tokens/s on H200. These figures describe the publisher’s generation setup. They do not measure a whole coding task with tool execution, retries, tests and review.

The llama.cpp pull-request author reports about 32 tokens/s on H200 for that implementation. The difference illustrates why a model-level speed claim cannot be assigned to every runtime. Neither value establishes the speed of a particular GGUF, Mac or consumer-GPU deployment.

For a coding comparison, use a repository with a known failing test, allow the same tools and require the repaired test plus relevant regression tests to pass. Count unsuccessful attempts. For logic or structured extraction, use cases with unambiguous answers. Record time to the first useful response and time to an accepted result alongside decode speed.

Clover’s Apache-2.0 weights can be deployed without a per-token license charge. Hardware allocation, power, storage and engineering time determine the local or rented-machine bill. The publisher offers an online-access application, rather than a public token tariff in the sources linked here.

For an assumed $2-per-hour GPU instance allocated for 30 minutes, compute is $1. If 20 accepted tasks finish in that allocation, compute averages $0.05 per accepted task. That example includes every attempt within the half hour, but excludes storage and other service charges. A fast decode rate only lowers cost if it reduces billed time or increases useful completed work.

For an owned machine averaging 400W for the same half hour, electricity use is 0.2 kWh. At an assumed $0.20/kWh, electricity is $0.04, before hardware ownership and other costs. Use measured whole-machine power and elapsed time; neither the 3B active count nor a weight-file size provides those measurements.