neuron-explorer

Overview

neuron-explorer is a steering-first neuron explorer for Qwen3.5-4B. It pushes every MLP neuron in a layer up and down on a few prompts, keeps every rollout, and lays the results out over a 3D view of the model. The explorer provides map, list, neuron, embeddings, and Claude chat panels for browsing neurons, steered rollouts, lenses, connections, and live inference.

Key Features

• 3D map of model layers with attention and MLP points per neuron, plus hover and click inspection of steer and logit lenses.
• Neuron list ranked by word-cluster mass against random-direction controls, with lens token search and activation filtering.
• Per-neuron steered rollouts per prompt and dose, including a three-column multi-turn steered chat with negative, zero, and positive steering.
• UMAP embeddings of a layer's rollouts colored by neuron, prompt, dose, or kind.
• Claude chat panel using the claude CLI with the explorer as an MCP server to read neurons, cones, and lenses, run live model operations, move views, and use file tools and a shell.
• Sweep tooling that records neuron stats, rollout text, token IDs, logprobs, top-k data, rank data, steer lenses, depth reports, embedding layouts, and connection caches.

Use Cases

• Investigating how individual MLP neurons affect model behavior by steering them across prompts and doses.
• Browsing model internals through layer maps, neuron lists, lenses, and connection views.
• Comparing rollout embeddings by neuron, prompt, dose, or kind to spot structure in steering results.
• Using an MCP-enabled Claude chat to query neurons and run live model experiments from the explorer.

Getting Started

• Install dependencies with uv sync.
• Start the explorer with uv run python explorer/server.py; it reads sweep runs from runs/ and per-layer stats from stats/.
• Generate sweep data on a CUDA 13 GPU pod by running sweep/setup_pod.sh, then sweep/neuron_stats.py and sweep/steer_sweep.py.

Deployment & Requirements

• The explorer loads Qwen3.5-4B locally on MPS, CUDA, or CPU.
• Sweep workflows run on a CUDA 13 GPU pod with the steering fork of vLLM.
• The Claude chat panel requires the claude CLI to be installed and logged in.
• The explorer reads runs from runs/ and per-layer stats from stats/, while explorer/cache/ ships with the repository.

Before You Adopt

• No recognizable license was detected; verify permission to use, modify, and distribute the project before adoption.
• Sweep workflows depend on a CUDA 13 GPU pod and the steering fork of vLLM.