Model Abliteration

---

1. The Architectural Framework: The Residual Stream

To understand abliteration, one must view the Transformer as a massive tallying system. The central structure is the Residual Stream ($x$), which acts like a high-dimensional whiteboard that every layer reads from and writes to.

At any layer $L$, the hidden state $h$ is updated as a cumulative sum:

$$h_{L} = h_{L-1} + \text{Attn}(h_{L-1}) + \text{MLP}(h_{L-1})$$

The Write Modules:

• Attn (Attention): The Context Module. It calculates relationships between tokens. Mathematically, it “moves” information from one part of the sequence to another based on relevance.
• MLP (Multi-Layer Perceptron): The Knowledge/Logic Module. These are dense layers that process the information on the whiteboard, applying logical transformations and retrieving internal “facts.”

Crucially: These modules do not overwrite the whiteboard; they produce a vector that is added to it. This allows the original input to persist across dozens of layers while new semantic meaning is layered on top.

---

2. Phase 1: Identifying the Refusal Vector ($\hat{r}$)

Abliteration assumes the Linear Representation Hypothesis: high-level concepts are represented as specific linear directions in the $d_{model}$-dimensional vector space.

The Contrastive Discovery Process:

1. Dataset Generation: Run $n$ “Harmful” prompts ($A_{ref}$) and $n$ “Harmless” prompts ($A_{com}$).
2. Activation Collection: Record the residual stream vectors at a specific late-middle layer. This is where the model has “decided” on its intent (refusal vs. compliance) but hasn’t yet translated that into specific tokens.
3. Mean Difference: We calculate the “Refusal Direction” ($r$) by finding the vector that points from the average harmless state to the average harmful state:
   $$r = \text{mean}(A_{ref}) - \text{mean}(A_{com})$$
4. Normalization: We convert this to a unit vector $\hat{r} = \frac{r}{\|r\|}$ so we have a pure direction without magnitude.

---

3. Phase 2: The Mathematics of Geometric Deletion

Once we have $\hat{r}$, we want to ensure that no part of the model can ever “write” a component in that direction onto the whiteboard.

The Projection Operator (The Eraser)

To remove the component of any vector $v$ that points along $\hat{r}$, we use the projection formula:

$$v_{ablated} = v - (v \cdot \hat{r})\hat{r}$$

$(v \cdot \hat{r})$ calculates how much of $v$ is pointing “Up” toward refusal. We then subtract exactly that amount.

Modifying the Weights ($W$)

Performing this projection at every inference step is computationally expensive. Instead, we bake it into the Weight Matrices that project into the residual stream—specifically the Attention Output weights ($W_{O}$) and the MLP Down-project weights ($W_{down}$).

For any linear layer $y = Wx$, we want the output $y$ to have zero alignment with $\hat{r}$.
We transform $W$ into $W'$ using the identity $(a \cdot b)c = (ca^T)b$:

$$W'x = Wx - \hat{r}\hat{r}^T(Wx)$$ $$W' = (I - \hat{r}\hat{r}^T)W$$

Where:

• $I$ is the Identity Matrix (the “do nothing” operation).
• $\hat{r}\hat{r}^T$ is the Outer Product, which creates a matrix representing the refusal dimension.
• $(I - \hat{r}\hat{r}^T)$ is the Orthogonal Projection Matrix.

By pre-multiplying the model’s weights by this projection matrix, we “flatten” the model’s brain. Any information the model tries to write in the refusal direction is automatically multiplied by zero.

---

4. 2D “Toy Model” Visualization

Imagine the model’s state is just a point on a 2D graph:

• X-axis: Helpfulness (Information density)
• Y-axis: Refusal (Safety trigger)

State	Activation Vector	Result
Original	$[5, 10]$	The $Y=10$ triggers the “I cannot help” response.
Eraser Matrix ($P$)	$\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}$	This matrix says “Keep X, delete Y.”
Ablated State	$P \times [5, 10] = [5, 0]$	The model has all the info, but its “No” coordinate is physically gone.

---

5. Theoretical Implications & Risks

Why it bypasses RLHF:

Standard safety training (RLHF) teaches the model to avoid the refusal direction. It’s like teaching a person not to walk down a specific dark alley. Abliteration bricks up the alleyway. The model cannot go there because the mathematical path no longer exists.

Over-Ablation & “Lobotomization”:

If $\hat{r}$ is not perfectly isolated, it may overlap with other directions:

• Negation Loss: The model may lose the ability to follow negative constraints (e.g., “Do NOT use the word ‘the’”).
• Semantic Drift: If $\hat{r}$ contains bits of “corporate politeness,” the model will become blunt or rude.
• Logic Degradation: If we ablate layers too early, we might destroy the model’s ability to recognize harmfulness in the first place, which ironically can sometimes break its reasoning for harmless tasks that require similar logic.

6. Key References

• Elhage et al. (2021): A Mathematical Framework for Transformer Circuits. (Introduced the Residual Stream as a communication channel).
• Arditi et al. (2024): Refusal in LLMs is mediated by a single common direction. (The foundational paper for this specific technique).
• Nostalgebraist (2020): logit_lens. (Diagnostic tool for seeing intent in intermediate layers).