Skip to main content
AdOpenFree logoPromote your productReach more potential users and drive product growth and revenue.Advertise
Favicon of Edge0

Edge0

Free Listing

Edge0 provides device-native foundation models and an open-source streaming MoE inference framework, running large sparse models such as 145B on a laptop and 35B on a phone with SSD-streamed weights, offline on-device processing, and zero

Visit Edge0-AI/Edge0

Overview

Edge0 is an open-source streaming MoE inference framework for running large sparse models on consumer hardware. It provides command-line, local API server, and Edge0 Chat app access, with current 145B and 120B latent-MoE models for a single NVIDIA GPU and a previous 35B release on the native engine.

Key Features

  • Runs large sparse mixture-of-experts models on consumer hardware using latent MoE and SSD-streamed expert weights.
  • Offers edge0-infer, a GGML-based unified inference engine with a shared C++ API, CLI, OpenAI-compatible local server, and chat app examples for desktop, Android, and iOS.
  • Supports on-device inference so sensitive data stays on the hardware without being sent to a cloud model for processing.
  • Enables offline operation and low-power, low-memory deployment across phones, laptops, earbuds, glasses, toys, and other devices.
  • Includes language and voice models such as Edge0-35B, Edge0-8B, Audio8-ASR-3B, and Audio8-TTS-0.6B.
  • Eliminates per-use charges and token costs by running inference on hardware you already ship.

Use Cases

  • Smart earbuds: an edge-cloud voice SDK embedded in earbuds for natural commands and live readouts with zero hardware changes.
  • Robotics and embodied AI: edge-cloud HRI SDK and robot chips plus edge voice models for natural-language dialogue, command execution, and offline or weak-network voice interaction.
  • Smart glasses: edge-cloud coordination keeps frequent commands on-device and complex tasks in the cloud to cut latency and cost.
  • Smart cockpit: a fully on-device cockpit for low power and high performance, with voice interaction accuracy of 93% and command response latency under 200 ms, keeping data in the vehicle.

Getting Started

  • For 145B and 120B, clone the Edge0 repository, build edge0-strata and edge0-infer using their build guides, and run edge0-infer serve --model /path/to/Edge0-145B-q3k from edge0-infer.
  • The serve command prints a local listening address and exposes OpenAI- and Anthropic-compatible endpoints.
  • Use edge0-infer chat --model /path/to/Edge0-145B-q3k for an interactive session, or run Edge0 Chat on Windows by choosing a model folder such as Edge0-145B-q3k.
  • Pass one model folder, not the folder that contains several models; edge0-infer identifies the model by model_name in model.json.

Deployment & Requirements

  • Latest 145B and 120B models run on Linux or Windows with an NVIDIA GPU and CUDA 13; build instructions target x64 or ARM64.
  • Platform support depends on model and engine: edge0-35b native engine supports Linux CPU, Windows CPU or Vulkan, macOS Apple Silicon Metal or CPU, iOS Metal, and Android CPU.
  • The 8B checkpoint is available, with native-engine support planned.

Before You Adopt

  • License: Apache-2.0. Review its terms before using, modifying, or distributing the project.
  • Model and platform support varies by engine; check the model-and-engine pairing before deployment.
  • The unified edge0-infer engine replaces the earlier Python framework and per-platform engines; older code is kept at a tag.
  • The 8B checkpoint is available, but native-engine support is planned rather than current.
  • A 16 GiB VRAM plus 20 GiB RAM reading is a test constraint, not a minimum system requirement.
  • Apple M2 Ultra Metal measurements do not establish macOS release availability.

Comments

Sign in to leave a comment.

More like Edge0

Favicon of kimi-k3-in-c

kimi-k3-in-c

Free ListingStars: 10K

2.78T-parameter Kimi K3 inference in portable C99

Language Models

Runs Kimi K3, a 2.78-trillion-parameter model, on a single CPU in 8.24 GB of RAM by streaming the model from disk instead of holding it in memory. Portable C99 with no BLAS, framework, or GPU.

Favicon of nanochat

nanochat

Free ListingStars: 60.1K

Minimal experimental harness for training LLMs

Language Models

nanochat is a minimal, hackable harness for training LLMs end to end on a single GPU node, covering tokenization, pretraining, finetuning, evaluation, and inference. It is pitched as the best ChatGPT that $100 can buy.

Favicon of Heretic

Heretic

Free ListingStars: 38.2K

Training-free language model modification

Language Models

Heretic is a training-free language model modification tool. It uses mathematical techniques and configurable scorers to steer model behavior, including removing censorship, suppressing refusals, reducing slop or sycophancy, and more.