Heretic

Overview

Heretic is a tool for training-free language model modification. It lets users modify language models they can run on their hardware without expensive training, using mathematical techniques to steer behavior based on contrasting example prompts and matching scorers. Refusal suppression is the default action.

Key Features

• Training-free modification of language models that can run on local hardware, with support for most dense models, many multimodal models, several MoE architectures, and many modern hybrid models.
• Default censorship removal and refusal suppression, with configuration options to reduce slop, reduce humor, reduce sycophancy, or alter emotional alignment.
• Configurable scorer plugins, including refusal rate and KL divergence by default, with support for custom scorers and co-optimization of multiple scores.
• Two built-in modifiers: Arbitrary-Rank Ablation (default) and directional ablation, also known as abliteration.
• Optional on-the-fly 4-bit quantization using bitsandbytes to reduce VRAM requirements at a slight accuracy cost.
• Reproducible model manifests and the Heretic Grimoire for cataloguing and reproducing published models.

Use Cases

• Removing censorship or suppressing refusals in a language model as the default processing action.
• Reducing slop, humor, or sycophancy, or altering emotional alignment, by selecting different configuration files.
• Writing custom scorer plugins to control and co-optimize multiple behavioral objectives.
• Saving, uploading to Hugging Face, chatting with, or benchmarking a processed model after a run.

Getting Started

• Prepare a Python 3.10+ environment with PyTorch 2.6+ installed as appropriate for your hardware.
• Run pip install -U heretic-llm.
• Run heretic Qwen/Qwen3-4B-Instruct-2507, replacing the model identifier with the model you want to process.
• By default, Heretic performs censorship removal (refusal suppression); choose another configuration file or write your own to do something else.

Deployment & Requirements

• Python 3.10+ environment with PyTorch 2.6+ installed as appropriate for your hardware.
• A language model that can run on your hardware.
• Optional 4-bit quantization with bitsandbytes can reduce VRAM requirements at a slight accuracy cost.

Before You Adopt

• License: AGPL-3.0. Review its terms before using, modifying, or distributing the project.
• Optional 4-bit quantization with bitsandbytes can lower VRAM requirements but may introduce a slight accuracy cost.
• Byte-for-byte reproduction of a processed model depends on an environment perfectly identical to the original, including hardware, drivers, operating system, and all package versions.