whisper.cpp

Overview

whisper.cpp provides high-performance inference of OpenAI's Whisper automatic speech recognition model in plain C/C++. It is a lightweight implementation that can be integrated into different platforms and applications, with support for CPU-only inference and acceleration across Apple Silicon, NVIDIA, AMD, Vulkan, OpenVINO, Ascend NPU, Moore Threads, and other targets.

Key Features

• Plain C/C++ implementation without dependencies
• Apple Silicon optimization via ARM NEON, Accelerate framework, Metal, and Core ML
• AVX intrinsics for x86, VSX intrinsics for POWER, and mixed F16/F32 precision
• Integer quantization support
• Support for CPU-only inference and acceleration via CUDA, ROCm, Vulkan, OpenVINO, CANN, MUSA, and other backends
• C-style API and voice activity detection (VAD)

Use Cases

• On-device and offline speech recognition on platforms such as iOS, Android, WebAssembly, Raspberry Pi, and desktop OSes
• Real-time audio input transcription from a microphone
• Building offline voice assistant applications
• Transcribing audio with timestamps, word-level timestamps, speaker segmentation, and karaoke-style movie generation

Getting Started

• Clone the whisper.cpp repository.
• Download a Whisper model converted in ggml format, for example by running sh ./models/download-ggml-model.sh base.en.
• Build the project with cmake -B build and cmake --build build -j --config Release.
• Transcribe a 16-bit WAV file with ./build/bin/whisper-cli -f samples/jfk.wav.
• For a quick demo, run make base.en.

Deployment & Requirements

• CMake is used to build the project.
• A Whisper model converted to the custom ggml format is required.
• The whisper-cli example only accepts 16-bit WAV files, so input may need conversion with ffmpeg or another tool.
• Optional acceleration backends have their own dependencies, such as CUDA, Vulkan, ROCm, OpenVINO, CANN, MUSA, Core ML, and ANEForge.
• Docker images are available for CPU, CUDA, MUSA, and Vulkan configurations.

Before You Adopt

• License: MIT. Review its terms before using, modifying, or distributing the project.
• The project supports inference only, not training or fine-tuning.
• The whisper-cli example only accepts 16-bit WAV files, so users may need to convert input audio before running it.
• Core ML execution has a slow first run because the ANE service compiles the model to a device-specific format.
• OpenVINO execution has a slow first run because the framework compiles the model to a device-specific blob.