Skip to main content
AdOpenFree logoPromote your productReach more potential users and drive product growth and revenue.Advertise

Tools tagged with "Quantization"

Favicon of ComfyUI

ComfyUI

Free ListingStars: 138.2K

Modular node-graph AI creation engine

Image Generation & Workflows

ComfyUI is a modular node-graph AI creation engine for generating images, video, 3D, audio, and text, with local and cloud options, an API, App Mode, and Comfy Agent for building and running workflows from natural-language descriptions.

Favicon of llama.cpp

llama.cpp

Free ListingStars: 129.5K

LLM inference in C/C++

Language Models

LLM inference in C/C++ with minimal setup and state-of-the-art performance on a wide range of hardware, locally and in the cloud.

Favicon of DeepSeek-V3

DeepSeek-V3

Free ListingStars: 104.5K

Mixture-of-Experts language model with 671B parameters

Language Models

DeepSeek-V3 is a Mixture-of-Experts language model with 671B total parameters and 37B activated per token. It supports a 128K context and is available in Base and Chat variants with open weights for local deployment and API use.

Favicon of vLLM

vLLM

Free ListingStars: 94.2K

Easy, fast, and cost-efficient LLM serving for everyone.

Inference Servers & APIs

vLLM is a high-throughput, memory-efficient inference and serving engine for LLMs. It aims to make LLM serving easy, fast, and cost-efficient for everyone.

Favicon of Unsloth

Unsloth

Free ListingStars: 76.8K

Open-source framework to run and train LLMs locally

Training & Fine-Tuning

Unsloth is an open-source framework for running and training LLMs. It provides a local UI to run and train LLMs and diffusion models, supporting GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

Favicon of whisper.cpp

whisper.cpp

Free ListingStars: 53.9K

High-performance C/C++ inference for OpenAI Whisper

Transcription & Dictation

whisper.cpp is a dependency-light C/C++ port of OpenAI's Whisper automatic speech recognition model, optimized for high-performance inference across CPUs, Apple Silicon, GPUs, NPUs, and edge platforms.

Favicon of llmfit

llmfit

Free ListingStars: 38.5K

Find LLMs that run on your hardware.

Evaluation & Benchmarks

llmfit detects CPU, RAM, GPU, VRAM, and accelerators, then ranks hundreds of open-source LLMs across quantizations by fit, speed, quality, and context so you can see what runs well on your machine.

Favicon of Heretic

Heretic

Free ListingStars: 38.2K

Training-free language model modification

Language Models

Heretic is a training-free language model modification tool. It uses mathematical techniques and configurable scorers to steer model behavior, including removing censorship, suppressing refusals, reducing slop or sycophancy, and more.

Favicon of SGLang

SGLang

Free ListingStars: 37.2K

High-performance LLM and multimodal model serving

Multimodal Models

SGLang is a high-performance serving framework for large language and multimodal models, designed for fast, scalable inference across diverse models and hardware.

Favicon of Qdrant

Qdrant

Free ListingStars: 34.8K

High-performance vector database and vector search engine

Vector Databases

Qdrant is a high-performance, massive-scale vector database and vector similarity search engine for next-generation AI applications. It stores, searches, and manages vectors with payload filtering. Also available as managed Qdrant Cloud.

Favicon of Strata

Strata

Free ListingStars: 31.2K

Run Qwen3.8-Flash-Next on your own PC

Chat Interfaces

Strata runs the 125B-parameter Qwen3.8-Flash-Next on consumer PCs, with one-click Windows and Linux installs, a local inference engine, OpenAI- and Anthropic-compatible localhost APIs, and optional image input.

Favicon of Qwen3

Qwen3

Free ListingStars: 27.7K

Large language model series by the Qwen team, Alibaba Cloud

Language Models

Qwen3 is the large language model series from Alibaba Cloud's Qwen team. It offers dense and Mixture-of-Experts models with thinking and non-thinking modes, multilingual, long-context, reasoning, coding, and agent capabilities.

Favicon of FreeToken

FreeToken

Free ListingStars: 15.1K

Datacenter-scale MoE model serving on your desktop

Edge & On-Device Inference

FreeToken is an edge-native Mixture-of-Experts serving engine that runs frontier-scale open-weight models locally on consumer and personal hardware, with adaptive CPU–GPU co-execution and OpenAI/Anthropic-compatible APIs.

Favicon of FlashMLA

FlashMLA

Free ListingStars: 13.1K

Optimized attention kernels for DeepSeek inference

Pipelines, Deployment & Compute

FlashMLA is DeepSeek's library of optimized attention kernels for inference on NVIDIA GPUs and Huawei Ascend NPUs, including sparse and dense attention kernels with FP8 or FP4 KV cache support.

Favicon of Edge0

Edge0

Free ListingStars: 11K

Device-native foundation models for on-device AI

Language Models

Edge0 provides device-native foundation models and an open-source streaming MoE inference framework, running large sparse models such as 145B on a laptop and 35B on a phone with SSD-streamed weights, offline on-device processing, and zero

Favicon of kimi-k3-in-c

kimi-k3-in-c

Free ListingStars: 10K

2.78T-parameter Kimi K3 inference in portable C99

Language Models

Runs Kimi K3, a 2.78-trillion-parameter model, on a single CPU in 8.24 GB of RAM by streaming the model from disk instead of holding it in memory. Portable C99 with no BLAS, framework, or GPU.

Favicon of TurboFieldfare

TurboFieldfare

Free ListingStars: 7.4K

Gemma 4 26B-A4B inference in about 2 GB of RAM

Chat Interfaces

TurboFieldfare is a Swift and Metal runtime for running Gemma 4 26B-A4B on Apple Silicon Macs with about 2 GB of RAM by streaming routed experts from SSD while keeping shared weights and KV cache resident.

Favicon of TensorFold

TensorFold

Free ListingStars: 1.5K

Native local LLM inference for Metal and CUDA.

Edge & On-Device Inference

TensorFold is an open-source LLM inference engine for Metal, CUDA, and Vulkan that runs open models locally on Apple Silicon and NVIDIA GPUs with native Metal and CUDA inference.

Favicon of openTPU

openTPU

Free ListingStars: 0.9K

Open-source AI accelerator: RTL, ISA, simulator, compiler

Quantization & Acceleration

openTPU is an open-source AI accelerator developed by AI, combining RTL, ISA, simulator, compiler, and profiler in one repository. It runs Qwen3, LFM2.5, and Qwen3.5 on a Kintex-7 PCIe card.

Ad
OpenFree logoPromote your product

Reach more potential users and drive product growth and revenue.

Advertise