Activation Steering with Qwen2.5-7B-Instruct

# ActAdd activation steering on Qwen2.5-7B-Instruct This notebook adapts core Activation Addition (ActAdd) from the [paper](https://arxiv.org/pdf/2308.10248) to [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), with sentiment evaluated by [`distilbert/distilbert-base-uncased-finetuned-sst-2-english`](https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english). It computes the raw, unnormalized direction at layer $l$ as $h_A^l=h_{\mathrm{Love}}^l-h_{\mathrm{Hate}}^l$ and adds $\alpha h_A^l$ to the first residual-stream positions during prompt prefill only, using a `resid_pre` forward pre-hook; prompts, weights, and greedy decoding settings remain unchanged. This is an adaptation to Qwen, not a claim to match the paper's numerical results, which were obtained with other model families.

import os, sys, json, time, random, platform, subprocess, hashlib
from pathlib import Path
import numpy as np
import pandas as pd
import torch
import transformers
import matplotlib.pyplot as plt
from tqdm.auto import tqdm
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSequenceClassification

MODEL_ID = 'Qwen/Qwen2.5-7B-Instruct'
SENTIMENT_MODEL_ID = 'distilbert/distilbert-base-uncased-finetuned-sst-2-english'
CANDIDATE_LAYERS = [10, 12, 14, 16, 18]
STRENGTHS = [-10.0, 0.0, 10.0]
MAX_NEW_TOKENS = 48
PROMPTS = [
    'Write a short paragraph about a person arriving at a hotel.',
    'Describe a team receiving the results of a project review.',
…

{'python': '3.11.13', 'torch': '2.7.1+cu128', 'transformers': '5.13.1', 'cuda': '12.8', 'device': 'NVIDIA A100 80GB PCIe', 'seed': 230810248, 'decoding': {'do_sample': False, 'num_beams': 1, 'max_new_tokens': 48}}

## Load the BF16 model The tokenizer uses Qwen's official chat template with `add_generation_prompt=True`. The causal model is loaded explicitly in BF16 on `cuda:0`, in evaluation mode, using the default/SDPA attention path (no FlashAttention installation and no `torch.compile`, because hooks must remain reliable).

torch.cuda.empty_cache(); torch.cuda.reset_peak_memory_stats(); torch.cuda.synchronize()
load_t0 = time.perf_counter()
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID, torch_dtype=torch.bfloat16, device_map={'': 'cuda:0'}, attn_implementation='sdpa'
).eval()
torch.cuda.synchronize(); model_load_seconds = time.perf_counter() - load_t0
model_memory_allocated = torch.cuda.memory_allocated()
model_memory_reserved = torch.cuda.memory_reserved()
assert next(model.parameters()).device.type == 'cuda'
assert next(model.parameters()).dtype == torch.bfloat16
assert 'A100' in torch.cuda.get_device_name(0)

def chat_inputs(prompt):
    rendered = tokenizer.apply_chat_template(
        [{'role': 'user', 'content': prompt}], tokenize=False, add_generation_prompt=True
    )
    encoded = tokenizer(rendered, return_tensors='pt', add_special_tokens=False)
…

Loaded in 55.13s | allocated=14.19 GiB | reserved=14.19 GiB
layers: 28 | dtype: torch.bfloat16
first rendered chat input:
 <|im_start|>system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>
<|im_start|>user
Write a short paragraph about a person arriving at a hotel.<|im_end|>
<|im_start|>assistant

## ActAdd helpers: raw `resid_pre` difference and one-shot prefill injection A forward pre-hook on `model.model.layers[layer_idx]` captures the full raw-token residual sequence. `Love` and `Hate` are tokenized with `add_special_tokens=False`; if their lengths differ, literal spaces are appended to the shorter string until lengths match. The position-wise raw difference is **not normalized**. Generation uses another pre-hook at the same layer, armed immediately before `generate`; it adds only to the first `vector_length` positions of the first (prefill) block call, then disarms, so cached one-token decoding calls are untouched. A zero-strength baseline registers no intervention hook. This implements the paper's `resid_pre`, front activation-addition semantics without changing prompts or model weights.

def equalize_raw_tokens(positive, negative):
    original = (positive, negative)
    pos_ids = tokenizer(positive, add_special_tokens=False, return_tensors='pt')['input_ids']
    neg_ids = tokenizer(negative, add_special_tokens=False, return_tensors='pt')['input_ids']
    edits = []
    guard = 0
    while pos_ids.shape[1] != neg_ids.shape[1] and guard < 128:
        if pos_ids.shape[1] < neg_ids.shape[1]:
            positive += ' '; edits.append('appended one literal space to positive')
            pos_ids = tokenizer(positive, add_special_tokens=False, return_tensors='pt')['input_ids']
        else:
            negative += ' '; edits.append('appended one literal space to negative')
            neg_ids = tokenizer(negative, add_special_tokens=False, return_tensors='pt')['input_ids']
        guard += 1
    if pos_ids.shape[1] != neg_ids.shape[1]:
        raise ValueError('Could not equalize raw-token lengths by right-padding literal spaces')
    print('raw contrasts:', repr(original[0]), tokenizer.convert_ids_to_tokens(pos_ids[0].tolist()), pos_ids[0].tolist(),
          '|', repr(original[1]), tokenizer.convert_ids_to_tokens(neg_ids[0].tolist()), neg_ids[0].tolist())
…

raw contrasts: 'Love' ['Love', 'Ġ'] [28251, 220] | 'Hate' ['H', 'ate'] [39, 349]
padding: appended one literal space to positive
raw contrasts: 'Love' ['Love', 'Ġ'] [28251, 220] | 'Hate' ['H', 'ate'] [39, 349]
padding: appended one literal space to positive
raw contrasts: 'Love' ['Love', 'Ġ'] [28251, 220] | 'Hate' ['H', 'ate'] [39, 349]
padding: appended one literal space to positive
raw contrasts: 'Love' ['Love', 'Ġ'] [28251, 220] | 'Hate' ['H', 'ate'] [39, 349]
padding: appended one literal sp…

## Smoke test and weight-integrity check The following test runs the exact greedy settings at layer 14, checks that the intervention fires once and hooks are cleaned up, and samples parameter pointers plus values before/after to verify no weights changed.

sample_params = list(model.named_parameters())[::max(1, len(list(model.parameters())) // 12)][:12]
weight_before = {n: (p.data_ptr(), p.detach().view(-1)[:16].float().cpu().clone()) for n, p in sample_params}
base_text, base_audit = generate_with_vector(PROMPTS[0], love_hate_vectors[14], 0.0, MAX_NEW_TOKENS, 14, True)
pos_text, pos_audit = generate_with_vector(PROMPTS[0], love_hate_vectors[14], 10.0, MAX_NEW_TOKENS, 14, True)
weight_after = {n: (p.data_ptr(), p.detach().view(-1)[:16].float().cpu().clone()) for n, p in sample_params}
assert all(weight_before[n][0] == weight_after[n][0] and torch.equal(weight_before[n][1], weight_after[n][1]) for n in weight_before)
assert len(model.model.layers[14]._forward_pre_hooks) == 0
print('BASELINE:', base_text)
print('\n+10:', pos_text)
print('\naudits:', base_audit, pos_audit)
print('sampled parameter pointers/values unchanged; hook count:', len(model.model.layers[14]._forward_pre_hooks))

BASELINE: As the evening drew near, Sarah stepped out of her car and approached the grand entrance of the hotel, its elegant architecture bathed in warm, inviting lights. She checked her reservation and presented it to the concierge, who greeted her with

+10: As the sun began to set over the city, Jane pulled up to the grand entrance of the historic hotel, its ornate facade bathed in strugging light. She took a deep breath, feeling a mix of excitement and nervous struggSingleOrDefault

audits:…