Gemma 4 26B in ~2 GB of RAM: the Swift runtime that streams experts from the SSD
In April we published a guide to running Gemma 4 on a Mac with Ollama, and the verdict on the 26B MoE variant was straightforward: with 24 GB of unified memory the model technically fits, but leaves macOS on the brink of collapse, and what you really want is 32 GB or more. It was correct at that time, and it was correct because of an assumption we all shared: weights have to live in RAM.
Andrey Mikhaylov published a runtime that breaks that assumption. TurboFieldfare is a Swift + Metal engine that runs the instruction-tuned checkpoint of Gemma 4 26B-A4B while keeping around 2 GB resident. It’s not a smaller quantization or a distilled variant: it’s the same 14.3 GB checkpoint, streamed from the SSD on the fly. It measures 5.1–6.3 tok/s on an 8 GB MacBook Air M2 and 31–35 tok/s on an M5 Pro.
Before you get excited: this requires macOS 26, Metal 4, and Swift 6.2+, plus about 15 GB of free disk space for the download and repack. If you’re on macOS 15 or earlier, this article is a preview, not a task list. The 8 GB refers to total system RAM, and the 8 GB Air M2 is the lowest configuration the author validated.
Why it works: the A4B in the name
Gemma 4 26B-A4B is a Mixture-of-Experts model. Twenty-six billion total parameters, but only about four billion are active for any given token: a router selects a small subset of experts per layer and the rest stays idle.
All mainstream runtimes load the 26B the same way, because you don’t know in advance which experts a token will need, and stalling the GPU to go fetch them is worse than paying the RAM cost. TurboFieldfare makes the opposite bet: it pays the I/O, saves the RAM.
What stays resident:
- The shared core — 1.35 GB of weights that every token touches: embeddings, attention, the routers themselves.
- The KV cache in FP16 — bounded circular buffers for the 25 sliding-window layers, linear storage for the 5 full-attention layers.
That’s your ~2 GB. Everything else — the expert weights, the overwhelming majority of the checkpoint — lives on the SSD and is fetched only when a token actually routes to it.
The per-token loop: Metal computes attention and router decisions from the resident weights, the CPU converts those decisions into a fetch plan, bounded-parallel pread calls bring the missing experts directly into buffers visible to Metal, and Metal combines the shared and routed outputs. An LFU cache of 16 slots per layer absorbs repeats.
Why pread and not mmap
This was the question that came up immediately on Hacker News, and it’s the right one: llama.cpp has been doing memory-mapping of weights for years, and the operating system’s page cache supposedly solves exactly this.
The author’s answer, in the thread: a cold expert costs about 2.8 ms with explicit bounded-parallel pread, versus about 10 ms when you let the system’s mmap handle it. End-to-end, on the same machine, that was the difference between 4 tok/s and 0.50 tok/s. Those are self-reported numbers from the author, but the mechanism is plausible: mmap faults serialize and the kernel has no idea which experts you’ll need, whereas the router tells you one layer ahead and you can issue the reads in parallel.
The other fact that makes the design work: 40% of experts repeat in the next token, and 57% within two tokens. That’s why a cache of just 16 slots per layer is enough to keep the SSD from becoming the bottleneck at each step. Language is repetitive, and the routing that follows it is too.
Installation
Everything that follows comes from the repo at release 0.1.1, Apache-2.0. Model weights are downloaded separately from Hugging Face under Google’s terms.
git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
Then install the model. The repack step is the smart part of the installer: it streams and rewrites the checkpoint to its own .gturbo layout without leaving the full download on disk, which matters if you’re running an 8 GB Air with 40 GB free.
swift run -c release TurboFieldfareRepack \
--output scratch/gemma4.gturbo \
--overwrite
If the download dies halfway through — and in a 15 GB pull it can happen — you resume it:
swift run -c release TurboFieldfareRepack \
--output scratch/gemma4.gturbo \
--overwrite \
--resume
To discard a corrupted partial state and start clean:
swift run -c release TurboFieldfareRepack \
--discard-partial \
--output scratch/gemma4.gturbo
And to confirm the result is intact before trusting any benchmark you run against it:
swift run -c release TurboFieldfareRepack \
--verify-install \
--input-gturbo scratch/gemma4.gturbo
Three ways to use it
Raw completion from the CLI — the fastest way to confirm it’s alive:
swift run -c release TurboFieldfareCLI \
--model scratch/gemma4.gturbo \
--prompt "The capital of France is" \
--max-new 64 \
--temperature 0
Chat with instructions, passing a messages file in the usual role/content form:
swift run -c release TurboFieldfareCLI \
--model scratch/gemma4.gturbo \
--messages-file messages.json
A local server that speaks OpenAI’s Chat Completions API, with streaming and function tools, bound to loopback:
swift build -c release --product TurboFieldfareServer
.build/release/TurboFieldfareServer \
--model scratch/gemma4.gturbo
It comes up at http://127.0.0.1:8080/v1, which means anything you already point to an OpenAI-compatible endpoint — your editor extension, a script, an agent framework — takes it as a drop-in by changing the base URL and putting in a throwaway API key.
There’s also a native Mac app, in SwiftUI/AppKit, with model download, loading, and the usual sampling controls:
.build/release/TurboFieldfareMac
The numbers, and the caveat that matters most
| Machine | Throughput |
|---|---|
| MacBook Air M2, 8 GB | 5.1–6.3 tok/s |
| M5 Pro | 31–35 tok/s |
All figures are from the author, measured on their hardware.
That sixfold difference isn’t mainly the chip. It’s the SSD. In the thread they noted that M5 disks read at about 6,323 MB/s while M4 ones hover around 2,031 MB/s — more than 3x. When your architecture streams expert weights at each token, storage bandwidth is your inference speed. Two Macs with the same RAM and different SSD generations won’t give you the same result, and that’s the variable worth checking before predicting what your machine will do.
At 5–6 tok/s on the Air you won’t enjoy a coding agent. And that’s fine: it’s roughly read speed, perfectly usable for text processing, extraction, JSON output, and batch jobs you launch and review later. Someone in the thread raised the obvious question about sustained overnight runs on a fanless chassis; nobody has posted thermal numbers yet, so treat long unattended jobs on an Air as unproven.
Another thing worth knowing before you commit the disk space: it’s text only — no images, audio or video, and no tool execution on the runtime side — and it’s a single-model-per-process by design. This isn’t a general-purpose inference framework, it’s a runtime built for one model. Exactly because of that it can do what it does.
The part that earns trust
The repo documents 103 measured results across kernels, caching, I/O, prefill and decode — including experiments that didn’t work. That’s rare, and it’s the reason this reads like engineering and not like a demo. You can see which cache policies lost, at what I/O depth it stopped helping, where prefill chunking mattered. When someone shows you their failures alongside their wins, the wins gain credibility, they don’t lose it.
Eight commits, one release, one open issue. It’s early software by any measure. But what it demonstrates won’t get un-demonstrated: for MoE models, the amount of RAM you have is a design decision, not a hard ceiling. Everyone doing local inference today has to explain why they’re still loading the full checkpoint.
If you want the RAM-based version of this decision — which Gemma 4 variant fits on your Mac the conventional way — our April guide remains the starting point: Gemma 4 en tu Mac con Ollama: Qué Variante Elegir Según tu RAM
And you? What’s the largest model you managed to run on your machine, and what did you have to give up to get there?