Skip to main content
AdOpenFree logoPromote your productReach more potential users and drive product growth and revenue.Advertise

Tools tagged with "CUDA"

Favicon of llama.cpp

llama.cpp

Free ListingStars: 129.5K

LLM inference in C/C++

Language Models

LLM inference in C/C++ with minimal setup and state-of-the-art performance on a wide range of hardware, locally and in the cloud.

Favicon of DeepSeek-V3

DeepSeek-V3

Free ListingStars: 104.5K

Mixture-of-Experts language model with 671B parameters

Language Models

DeepSeek-V3 is a Mixture-of-Experts language model with 671B total parameters and 37B activated per token. It supports a 128K context and is available in Base and Chat variants with open weights for local deployment and API use.

Favicon of vLLM

vLLM

Free ListingStars: 94.2K

Easy, fast, and cost-efficient LLM serving for everyone.

Inference Servers & APIs

vLLM is a high-throughput, memory-efficient inference and serving engine for LLMs. It aims to make LLM serving easy, fast, and cost-efficient for everyone.

Favicon of nanochat

nanochat

Free ListingStars: 60.1K

Minimal experimental harness for training LLMs

Language Models

nanochat is a minimal, hackable harness for training LLMs end to end on a single GPU node, covering tokenization, pretraining, finetuning, evaluation, and inference. It is pitched as the best ChatGPT that $100 can buy.

Favicon of whisper.cpp

whisper.cpp

Free ListingStars: 53.9K

High-performance C/C++ inference for OpenAI Whisper

Transcription & Dictation

whisper.cpp is a dependency-light C/C++ port of OpenAI's Whisper automatic speech recognition model, optimized for high-performance inference across CPUs, Apple Silicon, GPUs, NPUs, and edge platforms.

Favicon of Colibri

Colibri

Free ListingStars: 46.7K

Tiny engine, immense model.

Local Model Runners

Colibri is an open-source hierarchical inference engine that adapts LLM inference to your machine, coordinating CPU, GPU, RAM and storage while streaming mixture-of-experts weights from disk.

Favicon of VoxCPM

VoxCPM

Free ListingStars: 39.9K

Creative and expressive multilingual text-to-speech

Speech & Voice Generation

VoxCPM is a tokenizer-free text-to-speech system for multilingual speech generation, creative voice design, controllable cloning, and 48kHz studio-quality audio.

Favicon of llmfit

llmfit

Free ListingStars: 38.5K

Find LLMs that run on your hardware.

Evaluation & Benchmarks

llmfit detects CPU, RAM, GPU, VRAM, and accelerators, then ranks hundreds of open-source LLMs across quantizations by fit, speed, quality, and context so you can see what runs well on your machine.

Favicon of SGLang

SGLang

Free ListingStars: 37.2K

High-performance LLM and multimodal model serving

Multimodal Models

SGLang is a high-performance serving framework for large language and multimodal models, designed for fast, scalable inference across diverse models and hardware.

Favicon of DwarfStar

DwarfStar

Free ListingStars: 24K

Local inference engine for DeepSeek, GLM, and Qwen

Inference Servers & APIs

DwarfStar is a small native inference engine for running selected DeepSeek, GLM, and Qwen models on consumer hardware with Metal, CUDA, and ROCm support. It uses the project's own GGUF files and is not a general GGUF runner.

Favicon of Hunyuan3D

Hunyuan3D

Free ListingStars: 15K

All-in-one AI creation platform for 3D content

3D Generation & Design

Hunyuan3D is an all-in-one AI creation platform for 3D content. It supports text-to-3D, image-to-3D, 3D animation, texture generation, sketch-to-3D, and 3D characters, with low-poly advantages. It targets high-resolution textured 3D assets

Favicon of FlashMLA

FlashMLA

Free ListingStars: 13.1K

Optimized attention kernels for DeepSeek inference

Pipelines, Deployment & Compute

FlashMLA is DeepSeek's library of optimized attention kernels for inference on NVIDIA GPUs and Huawei Ascend NPUs, including sparse and dense attention kernels with FP8 or FP4 KV cache support.

Favicon of LMCache

LMCache

Free ListingStars: 12.3K

KV cache management layer for scalable LLM inference

Context Management

LMCache is an open-source KV cache management layer for LLM inference that stores, compresses, searches, and reuses KV cache across GPU, CPU, local storage, and external backends to reduce repeated prefill and improve time-to-first-token.

Favicon of Kornia

Kornia

Free ListingStars: 11.6K

Differentiable computer vision for PyTorch

Image Editing & Enhancement

Kornia is a differentiable computer vision library for PyTorch, offering batched, GPU-accelerated operators for image processing, geometry, feature matching, augmentation, and pretrained models for spatial AI.

Favicon of Edge0

Edge0

Free ListingStars: 11K

Device-native foundation models for on-device AI

Language Models

Edge0 provides device-native foundation models and an open-source streaming MoE inference framework, running large sparse models such as 145B on a laptop and 35B on a phone with SSD-streamed weights, offline on-device processing, and zero

Favicon of Kev

Kev

Free ListingStars: 9.6K

Small Jev-like decision models to train and run yourself.

Language Models

Kev is a family of small decision models built on Qwen3.5 and Qwen3.8, based on Jev's architecture. Use pretrained weights or train your own, with yes/no, multiple-choice, and rating questions in one request.

Favicon of DeepInfer

DeepInfer

Free ListingStars: 7.9K

High-performance inference framework for DeepSeek models

Inference Servers & APIs

DeepInfer is a high-performance, large-scale inference framework for DeepSeek models. This open-source release provides offline inference with optimized kernels, parallelism, scheduling, deterministic sampling, and structured outputs.

Favicon of CUDA Rust

CUDA Rust

Free ListingStars: 3.9K

NVIDIA's CUDA platform for Rust

Pipelines, Deployment & Compute

CUDA Rust is NVIDIA's CUDA platform for Rust, with host-side runtime crates and device-side tools for writing GPU kernels in idiomatic Rust using SIMT and Tile models.

Favicon of ATLAS

ATLAS

Free ListingStars: 2.1K

Local coding agent for compact open models

Code Assistants & Editors

ATLAS is a local coding agent from Inferstep, an open lab challenging assumptions on scale. It adds planning, verification, and repair around compact open models so software work can run on local hardware.

Favicon of TensorFold

TensorFold

Free ListingStars: 1.5K

Native local LLM inference for Metal and CUDA.

Edge & On-Device Inference

TensorFold is an open-source LLM inference engine for Metal, CUDA, and Vulkan that runs open models locally on Apple Silicon and NVIDIA GPUs with native Metal and CUDA inference.

Favicon of neuron-explorer

neuron-explorer

Free ListingStars: 1.3K

Steering-first neuron explorer for Qwen3.5-4B

Language Models

A steering-first explorer for Qwen3.5-4B that pushes every MLP neuron, keeps every rollout, and lays the results out over a 3D view of the model.

Favicon of CorridorKey 2.0

CorridorKey 2.0

Free ListingStars: 1.1K

One-click chroma keying for green and blue screens

Video Editing & Animation

CorridorKey 2.0 is a one-click chroma keying tool for green and blue screens that runs locally in VFX and editing software, with plugins for Resolve, Nuke, After Effects, Premiere and Blender; one purchase gets every plugin, no

Ad
OpenFree logoPromote your product

Reach more potential users and drive product growth and revenue.

Advertise