TurboFieldfare
TurboFieldfare is a Swift and Metal runtime for running Gemma 4 26B-A4B on Apple Silicon Macs with about 2 GB of RAM by streaming routed experts from SSD while keeping shared weights and KV cache resident.
Visit drumih/turbo-fieldfareOverview
TurboFieldfare is a model-specific Swift and Metal runtime and native Mac app for instruction-tuned Gemma 4 26B-A4B. It keeps the shared core and FP16 KV cache in memory and streams only the routed experts needed for each token from SSD, allowing the model to run on Apple Silicon Macs with 8 GB of RAM. The project also provides a command-line interface, a streaming installer and verifier, and an experimental loopback OpenAI-compatible Chat Completions server.
Key Features
- Runs Gemma 4 26B-A4B IT on Apple Silicon Macs with about 2 GB for weights and a 4K KV cache by streaming routed experts from SSD.
- Includes a native Mac app, command-line interface, streaming model installer and verifier, and loopback OpenAI-compatible server.
- Uses a custom Swift and Metal runtime with 4-bit MLX affine weights, an 8-bit router, and custom Metal kernels for quantized GEMV, attention, MoE, normalization, RoPE, and sampling.
- Uses chunked prompt prefill and token-by-token generation with FP16 KV storage.
- Supports optional image input through a separately installed companion vision pack on M2 or newer Apple Silicon Macs.
Use Cases
- Local instruction chat and text generation on low-memory Apple Silicon Macs, including 8 GB models.
- Command-line chat and raw completion using an installed .gturbo model.
- Testing clients against a local OpenAI-compatible Chat Completions endpoint, including streaming and function-tool declarations handled by the client.
- Image question answering when the optional vision pack is installed on supported M2 or newer hardware.
Getting Started
- Clone the repository, run swift build -c release, then launch .build/release/TurboFieldfareMac. In the app, choose Download Model to fetch and repack the pinned model, then choose Load Model, enter a prompt, and choose Generate.
Deployment & Requirements
- Apple Silicon Mac; the validated target is an 8 GB M2 MacBook Air.
- macOS 26 with Metal 4.
- Apple Command Line Tools or Xcode with Swift 6.2+ and the macOS 26 SDK or newer.
- Enough free storage for the about 14.3 GB model installation and an internet connection for the first model install.
- The package is arm64-only; older macOS and Metal versions are not supported.
Before You Adopt
- License: Apache-2.0. Review its terms before using, modifying, or distributing the project.