llmfit

Overview

llmfit is a terminal-first tool for matching local large language models to a machine's CPU, system RAM, discrete or integrated GPU, VRAM, and unified memory. It inspects hardware, scores catalog models across memory fit, estimated speed, quality, and context, and presents recommendations through an interactive TUI, classic CLI, web dashboard, or HTTP API.

Key Features

• Hardware auto-detection for CPU cores, system RAM, discrete/integrated GPUs, VRAM, and unified memory across NVIDIA CUDA, Apple Silicon, AMD ROCm, and Intel OneAPI.
• Model compatibility scoring for parameter counts, context lengths, and quantization formats including GGUF, AWQ, GPTQ, and EXL2, with projected memory footprints and tokens-per-second estimates.
• Interactive TUI plus a feature-rich web dashboard for browsing, filtering, planning, simulating hardware, downloads, and benchmarks.
• REST API endpoints /api/v1/system and /api/v1/models for integration into orchestrators, dashboards, and automated deployment pipelines.
• Support for multi-GPU setups, MoE architectures, dynamic quantization selection, speed estimation, and local runtime providers such as Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio.
• Community benchmark sharing via llmfit bench --share, with locally saved runs replacing estimates in the fit table and merged submissions shipped in later releases.

Use Cases

• Choosing which local LLM and quantization will fit a laptop, workstation, or multi-GPU system before downloading a model.
• Planning deployments by estimating storage needs for a set of runnable models and checking hardware fit.
• Feeding JSON recommendations into scripts, agents, or orchestrators that select models automatically.
• Benchmarking real tokens-per-second and TTFT on a local runtime and sharing results with the community.

Getting Started

• Install on macOS or Linux with Homebrew: brew install AlexsJones/llmfit/llmfit; Windows users can use winget install --exact --id AlexsJones.llmfit or scoop install llmfit.
• Alternatively install with uv: uv tool install -U llmfit, or run without installing with uvx llmfit.
• Launch the interactive TUI by running llmfit, or use commands such as llmfit recommend --json, llmfit fit, llmfit bench, llmfit doctor, and llmfit serve.
• Run the container with docker run -it --rm ghcr.io/alexsjones/llmfit --tui for the TUI, or docker run -d -p 8787:8787 ghcr.io/alexsjones/llmfit serve for the Web UI and API server.

Deployment & Requirements

• Runs on macOS (Apple Silicon and Intel), Linux (x86_64 and ARM64), and Windows (x86_64).
• A multi-architecture Docker image is provided at ghcr.io/alexsjones/llmfit for interactive CLI/TUI and headless Web UI/API server modes.
• The API and web server can be started with llmfit serve --host 0.0.0.0 --port 8787.
• Building from source uses git clone, cd llmfit, and cargo build --release; the binary is at target/release/llmfit.

Before You Adopt

• License: MIT. Review its terms before using, modifying, or distributing the project.