Reach more potential users and drive product growth and revenue.
AdvertiseTools tagged with "CUDA"
llama.cpp
Free ListingStars: 129.5KLLM inference in C/C++
LLM inference in C/C++ with minimal setup and state-of-the-art performance on a wide range of hardware, locally and in the cloud.
DeepSeek-V3
Free ListingStars: 104.5KMixture-of-Experts language model with 671B parameters
DeepSeek-V3 is a Mixture-of-Experts language model with 671B total parameters and 37B activated per token. It supports a 128K context and is available in Base and Chat variants with open weights for local deployment and API use.
vLLM
Free ListingStars: 94.2KEasy, fast, and cost-efficient LLM serving for everyone.
vLLM is a high-throughput, memory-efficient inference and serving engine for LLMs. It aims to make LLM serving easy, fast, and cost-efficient for everyone.
nanochat
Free ListingStars: 60.1KMinimal experimental harness for training LLMs
nanochat is a minimal, hackable harness for training LLMs end to end on a single GPU node, covering tokenization, pretraining, finetuning, evaluation, and inference. It is pitched as the best ChatGPT that $100 can buy.
whisper.cpp
Free ListingStars: 53.9KHigh-performance C/C++ inference for OpenAI Whisper
whisper.cpp is a dependency-light C/C++ port of OpenAI's Whisper automatic speech recognition model, optimized for high-performance inference across CPUs, Apple Silicon, GPUs, NPUs, and edge platforms.
Colibri
Free ListingStars: 46.7KTiny engine, immense model.
Colibri is an open-source hierarchical inference engine that adapts LLM inference to your machine, coordinating CPU, GPU, RAM and storage while streaming mixture-of-experts weights from disk.
VoxCPM
Free ListingStars: 39.9KCreative and expressive multilingual text-to-speech
VoxCPM is a tokenizer-free text-to-speech system for multilingual speech generation, creative voice design, controllable cloning, and 48kHz studio-quality audio.
llmfit
Free ListingStars: 38.5KFind LLMs that run on your hardware.
llmfit detects CPU, RAM, GPU, VRAM, and accelerators, then ranks hundreds of open-source LLMs across quantizations by fit, speed, quality, and context so you can see what runs well on your machine.
SGLang
Free ListingStars: 37.2KHigh-performance LLM and multimodal model serving
SGLang is a high-performance serving framework for large language and multimodal models, designed for fast, scalable inference across diverse models and hardware.
DwarfStar
Free ListingStars: 24KLocal inference engine for DeepSeek, GLM, and Qwen
DwarfStar is a small native inference engine for running selected DeepSeek, GLM, and Qwen models on consumer hardware with Metal, CUDA, and ROCm support. It uses the project's own GGUF files and is not a general GGUF runner.
Hunyuan3D
Free ListingStars: 15KAll-in-one AI creation platform for 3D content
Hunyuan3D is an all-in-one AI creation platform for 3D content. It supports text-to-3D, image-to-3D, 3D animation, texture generation, sketch-to-3D, and 3D characters, with low-poly advantages. It targets high-resolution textured 3D assets
FlashMLA
Free ListingStars: 13.1KOptimized attention kernels for DeepSeek inference
FlashMLA is DeepSeek's library of optimized attention kernels for inference on NVIDIA GPUs and Huawei Ascend NPUs, including sparse and dense attention kernels with FP8 or FP4 KV cache support.
LMCache
Free ListingStars: 12.3KKV cache management layer for scalable LLM inference
LMCache is an open-source KV cache management layer for LLM inference that stores, compresses, searches, and reuses KV cache across GPU, CPU, local storage, and external backends to reduce repeated prefill and improve time-to-first-token.
Kornia
Free ListingStars: 11.6KDifferentiable computer vision for PyTorch
Kornia is a differentiable computer vision library for PyTorch, offering batched, GPU-accelerated operators for image processing, geometry, feature matching, augmentation, and pretrained models for spatial AI.
Edge0
Free ListingStars: 11KDevice-native foundation models for on-device AI
Edge0 provides device-native foundation models and an open-source streaming MoE inference framework, running large sparse models such as 145B on a laptop and 35B on a phone with SSD-streamed weights, offline on-device processing, and zero
Kev
Free ListingStars: 9.6KSmall Jev-like decision models to train and run yourself.
Kev is a family of small decision models built on Qwen3.5 and Qwen3.8, based on Jev's architecture. Use pretrained weights or train your own, with yes/no, multiple-choice, and rating questions in one request.
DeepInfer
Free ListingStars: 7.9KHigh-performance inference framework for DeepSeek models
DeepInfer is a high-performance, large-scale inference framework for DeepSeek models. This open-source release provides offline inference with optimized kernels, parallelism, scheduling, deterministic sampling, and structured outputs.
CUDA Rust
Free ListingStars: 3.9KNVIDIA's CUDA platform for Rust
CUDA Rust is NVIDIA's CUDA platform for Rust, with host-side runtime crates and device-side tools for writing GPU kernels in idiomatic Rust using SIMT and Tile models.
ATLAS
Free ListingStars: 2.1KLocal coding agent for compact open models
ATLAS is a local coding agent from Inferstep, an open lab challenging assumptions on scale. It adds planning, verification, and repair around compact open models so software work can run on local hardware.
TensorFold
Free ListingStars: 1.5KNative local LLM inference for Metal and CUDA.
TensorFold is an open-source LLM inference engine for Metal, CUDA, and Vulkan that runs open models locally on Apple Silicon and NVIDIA GPUs with native Metal and CUDA inference.
neuron-explorer
Free ListingStars: 1.3KSteering-first neuron explorer for Qwen3.5-4B
A steering-first explorer for Qwen3.5-4B that pushes every MLP neuron, keeps every rollout, and lays the results out over a 3D view of the model.
CorridorKey 2.0
Free ListingStars: 1.1KOne-click chroma keying for green and blue screens
CorridorKey 2.0 is a one-click chroma keying tool for green and blue screens that runs locally in VFX and editing software, with plugins for Resolve, Nuke, After Effects, Premiere and Blender; one purchase gets every plugin, no

