Tools tagged with "Quantization"

LLM inference in C/C++ with minimal setup and state-of-the-art performance on a wide range of hardware, locally and in the cloud.

DeepSeek-V3 is a Mixture-of-Experts language model with 671B total parameters and 37B activated per token. It supports a 128K context and is available in Base and Chat variants with open weights for local deployment and API use.

Unsloth is an open-source framework for running and training LLMs. It provides a local UI to run and train LLMs and diffusion models, supporting GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

Qdrant is a high-performance, massive-scale vector database and vector similarity search engine for next-generation AI applications. It stores, searches, and manages vectors with payload filtering. Also available as managed Qdrant Cloud.

Qwen3 is the large language model series from Alibaba Cloud's Qwen team. It offers dense and Mixture-of-Experts models with thinking and non-thinking modes, multilingual, long-context, reasoning, coding, and agent capabilities.