TurboFieldfare: run Gemma 4 with a smaller memory budget

TurboFieldfare is a Swift and Metal inference runtime for the instruction-tuned Gemma 4 26B-A4B on Apple Silicon. It keeps a shared core in memory and streams selected experts from SSD for each token. Download the Gemma 4 model separately; TurboFieldfare provides the runtime.

The maintainer reports roughly 2GB of runtime footprint with a 4K KV cache on an 8GB M2 MacBook Air. That saving comes with storage reads and waiting time. Use it when trying the supported model on a Mac you already own; measure the complete task before buying hardware around a short decode benchmark.

Check the Mac, model and storage first

The documented platform is Apple Silicon, macOS 26 or later, Metal 4 and Swift 6.2. The main text model uses MLX affine 4-bit weights with group size 64 and an 8-bit router. Its roughly 14.3GB on-disk size is not a 2GB download. An optional image pack adds about 1.1GB; use the text path first if images are unnecessary.

The shared core occupies about 1.35GB. KV cache, expert-cache slots, runtime state, the app and macOS also use memory. Longer contexts and different cache settings change the footprint. Keep enough free SSD capacity for installation and the chosen model, and use the exact artifact and repacking instructions in the project README.

The runtime code is Apache-2.0; the Gemma model has its own terms. Read the project’s third-party notices before redistributing a packaged application or weights.

Build and try one short prompt

Follow the source installation instructions with an appropriate Xcode toolchain:

git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac

The first build downloads Swift package dependencies. In the app, choose Download Model, wait for the pinned download and repacking to finish, then choose Load Model and generate a short response. Save the runtime revision and model path with the result. Installation, repacking and loading time belong in your first-session budget.

A reported Swift 6.4 build failure concerns a CommandLineTools-only toolchain missing SwiftUIMacros. Check which toolchain is active if the app build produces that error; a successful CLI or kernel build does not establish that the SwiftUI app builds under the same toolchain.

The project also has a CLI and an experimental loopback OpenAI-compatible server. Start with its local server guide when connecting a client. The app and CLI do not expose or execute tools; the server’s function-tool interface is a separate integration. Do not assume a chat window is a coding agent with file and command access.

Read the measurements with their workload

The maintainer’s benchmark record reports these M2 results. Each workload ran once in a fresh process with a warm but uncontrolled file cache. Decode excludes installation, loading and prompt prefill.

These points explain why a small memory footprint can still mean a long wait for a repository-sized prompt. They do not measure sustained long generation or establish a guaranteed speed range for all M-series Macs. The same record includes a 24GB M5 Pro comparison; its hardware, prompt protocol and repeated-run method differ from the M2 rows.

Community issue 190 reports a stray token in a long-context greedy response on an M5 MacBook with 16GB, at recorded revisions. The issue is now closed; the cited observation concerns those historical revisions. Inspect names, numbers and text copied from earlier context in your own task. A correct short completion does not validate a much longer conversation.

Community-submitted Mac runs

The community protocol uses three real-generation-v1 tasks: a short explanation, medium design review and long synthesis. Its settings are temperature 0.2, Top-K 64, Top-P 0.95, a 4,096-token context and up to 1,024 generated tokens, with fixed seeds 20260721/22/23. Each submitter discarded a warmup, then ran each task once in a fresh process. The table reports input → generated tokens / decode tokens per second; it excludes setup, loading and prompt prefill.

The M1 Air report, opened August 10, uses macOS 26.6.1 and revision 417f38934c7590db15e7a2baa35f3cea05828dda. The Neo report, opened August 4, uses macOS 26.5.2 and f43822ec022d675ac0acc9532601d64a9be68763; it explicitly did not capture peak in-flight resident memory. The M1 Pro report, opened August 1, uses macOS 26.5.2 and revision 1859181, without TTFT, prefill timing or RSS. Opening dates are not exact measurement timestamps.

Treat these one-run submissions as configurations to try, not a controlled hardware ranking. Generated lengths and routing work differ, so test your own workload if sustained or multi-turn performance matters.

Compare complete session cost

On a Mac you already own, separate the purchase cost from the time and energy spent running the model. Measure wall power over installation, loading, prefill and generation. For illustration, an assumed 30W machine running for one hour uses 0.03kWh; at $0.20/kWh, electricity is $0.006. Those assumptions are not measurements of the M2 benchmark or a complete ownership cost.

For a useful comparison, send the same prompt and output budget through TurboFieldfare and a runtime that fits the supported model in memory. Count setup, failed runs and review time, then compare cost per accepted answer. SSD traffic is part of this design; the cited measurements do not establish a long-term drive-endurance cost.

Use the local inference directory to compare runtime strategies and the running-cost notes for electricity, hardware and allocated time. Choose a short representative task before increasing context or connecting an agent client.

Sources checked October 26, 2026: maintainer setup and architecture, benchmark protocol, community protocol and linked submissions, issue 185 and issue 190.