Reach more potential users and drive product growth and revenue.
AdvertiseTools tagged with "Quantization"
ComfyUI
Free ListingStars: 138.2KModular node-graph AI creation engine
ComfyUI is a modular node-graph AI creation engine for generating images, video, 3D, audio, and text, with local and cloud options, an API, App Mode, and Comfy Agent for building and running workflows from natural-language descriptions.
llama.cpp
Free ListingStars: 129.5KLLM inference in C/C++
LLM inference in C/C++ with minimal setup and state-of-the-art performance on a wide range of hardware, locally and in the cloud.
DeepSeek-V3
Free ListingStars: 104.5KMixture-of-Experts language model with 671B parameters
DeepSeek-V3 is a Mixture-of-Experts language model with 671B total parameters and 37B activated per token. It supports a 128K context and is available in Base and Chat variants with open weights for local deployment and API use.
vLLM
Free ListingStars: 94.2KEasy, fast, and cost-efficient LLM serving for everyone.
vLLM is a high-throughput, memory-efficient inference and serving engine for LLMs. It aims to make LLM serving easy, fast, and cost-efficient for everyone.
Unsloth
Free ListingStars: 76.8KOpen-source framework to run and train LLMs locally
Unsloth is an open-source framework for running and training LLMs. It provides a local UI to run and train LLMs and diffusion models, supporting GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.
whisper.cpp
Free ListingStars: 53.9KHigh-performance C/C++ inference for OpenAI Whisper
whisper.cpp is a dependency-light C/C++ port of OpenAI's Whisper automatic speech recognition model, optimized for high-performance inference across CPUs, Apple Silicon, GPUs, NPUs, and edge platforms.
llmfit
Free ListingStars: 38.5KFind LLMs that run on your hardware.
llmfit detects CPU, RAM, GPU, VRAM, and accelerators, then ranks hundreds of open-source LLMs across quantizations by fit, speed, quality, and context so you can see what runs well on your machine.
Heretic
Free ListingStars: 38.2KTraining-free language model modification
Heretic is a training-free language model modification tool. It uses mathematical techniques and configurable scorers to steer model behavior, including removing censorship, suppressing refusals, reducing slop or sycophancy, and more.
SGLang
Free ListingStars: 37.2KHigh-performance LLM and multimodal model serving
SGLang is a high-performance serving framework for large language and multimodal models, designed for fast, scalable inference across diverse models and hardware.
Qdrant
Free ListingStars: 34.8KHigh-performance vector database and vector search engine
Qdrant is a high-performance, massive-scale vector database and vector similarity search engine for next-generation AI applications. It stores, searches, and manages vectors with payload filtering. Also available as managed Qdrant Cloud.
Strata
Free ListingStars: 31.2KRun Qwen3.8-Flash-Next on your own PC
Strata runs the 125B-parameter Qwen3.8-Flash-Next on consumer PCs, with one-click Windows and Linux installs, a local inference engine, OpenAI- and Anthropic-compatible localhost APIs, and optional image input.
Qwen3
Free ListingStars: 27.7KLarge language model series by the Qwen team, Alibaba Cloud
Qwen3 is the large language model series from Alibaba Cloud's Qwen team. It offers dense and Mixture-of-Experts models with thinking and non-thinking modes, multilingual, long-context, reasoning, coding, and agent capabilities.
FreeToken
Free ListingStars: 15.1KDatacenter-scale MoE model serving on your desktop
FreeToken is an edge-native Mixture-of-Experts serving engine that runs frontier-scale open-weight models locally on consumer and personal hardware, with adaptive CPU–GPU co-execution and OpenAI/Anthropic-compatible APIs.
FlashMLA
Free ListingStars: 13.1KOptimized attention kernels for DeepSeek inference
FlashMLA is DeepSeek's library of optimized attention kernels for inference on NVIDIA GPUs and Huawei Ascend NPUs, including sparse and dense attention kernels with FP8 or FP4 KV cache support.
Edge0
Free ListingStars: 11KDevice-native foundation models for on-device AI
Edge0 provides device-native foundation models and an open-source streaming MoE inference framework, running large sparse models such as 145B on a laptop and 35B on a phone with SSD-streamed weights, offline on-device processing, and zero
kimi-k3-in-c
Free ListingStars: 10K2.78T-parameter Kimi K3 inference in portable C99
Runs Kimi K3, a 2.78-trillion-parameter model, on a single CPU in 8.24 GB of RAM by streaming the model from disk instead of holding it in memory. Portable C99 with no BLAS, framework, or GPU.
TurboFieldfare
Free ListingStars: 7.4KGemma 4 26B-A4B inference in about 2 GB of RAM
TurboFieldfare is a Swift and Metal runtime for running Gemma 4 26B-A4B on Apple Silicon Macs with about 2 GB of RAM by streaming routed experts from SSD while keeping shared weights and KV cache resident.
TensorFold
Free ListingStars: 1.5KNative local LLM inference for Metal and CUDA.
TensorFold is an open-source LLM inference engine for Metal, CUDA, and Vulkan that runs open models locally on Apple Silicon and NVIDIA GPUs with native Metal and CUDA inference.
openTPU
Free ListingStars: 0.9KOpen-source AI accelerator: RTL, ISA, simulator, compiler
openTPU is an open-source AI accelerator developed by AI, combining RTL, ISA, simulator, compiler, and profiler in one repository. It runs Qwen3, LFM2.5, and Qwen3.5 on a Kintex-7 PCIe card.

