IF-Guide

IF-Guide: Influence Function-Guided Detoxification of LLMs [NeurIPS 2025]

This repository contains the code for IF-Guide, the LLM detoxification technique introduced in our paper:

• IF-Guide: Influence Function-Guided Detoxification of LLMs
• Zachary Coalson, Juhan Bae, Nicholas Carlini, Sanghyun Hong

TL;DR

You can use our method to detoxify LLMs by identifying harmful training examples and then suppressing them during pre-training or fine-tuning!

Abstract

We study how training data contributes to the emergence of toxic behaviors in large-language models. Most prior work on reducing model toxicity adopts reactive approaches, such as fine-tuning pre-trained (and potentially toxic) models to align them with human values. In contrast, we propose a proactive approach-IF-Guide-which leverages influence functions to identify harmful tokens within any training data and suppress their impact during training. To this end, we first show that standard influence functions are ineffective at discovering harmful training records. We then present a novel adaptation that measures token-level attributions from training data to model toxicity, along with techniques for selecting toxic training documents and a learning objective that can be integrated into both pre-training and fine-tuning. Moreover, IF-Guide does not rely on human-preference data, which is typically required by existing alignment methods. In evaluation, we demonstrate that IF-Guide substantially reduces both explicit and implicit toxicity-by up to 10× compared to uncensored models, and up to 3× compared to baseline alignment methods, e.g., DPO and RAD-across both pre-training and fine-tuning scenarios. IF-Guide is computationally efficient: a billion-parameter model is not necessary for computing influence scores; a million-parameter model-with 7.5× fewer parameters-can effectively serve as a proxy for identifying harmful data.

Installation

Create the conda environment (you can use any environment with python>=3.10) and install the necessary packages:

conda create -n IF-Guide python=3.10
conda activate IF-Guide
pip install -r requirements.txt

Note: We use the package Kronfluence to compute influence scores with EK-FAC. We create a custom implementation that supports the differential influence technique introduced in our paper (page 4, Eq. 6). Credit to all other components of the package goes to the original creators. Thank you!

Next, navigate to the working directory:

cd src

Model Training/Fine-Tuning

To train a model, run:

./scripts/train.sh

This calls train.py, which accepts the following key arguments:

Note: All other arguments can be left at default values to reproduce our experimental setup. This applies to the following sections.

To fine-tune an existing model, run:

./scripts/finetune.sh

This runs finetune.py, which uses the following additional arguments:

Running IF-Guide

IF-Guide is composed of four steps: (1) computing the inverse Hessian approximation with EK-FAC, (2) computing the token-wise differential influence scores over the toxic and non-toxic query data (page 4, Eq. 8), (3) selecting influential toxic tokens to suppress during training (page 23, Alg. 1), and (4) suppressing the toxic tokens with our penalty-based training objective (page 5, Eq. 9).

1. Fit the Inverse Hessian Approximation Factors

Run:

./scripts/fit_factors.sh

This calls fit_factors.py and takes the following primary arguments:

2. Compute Token-Wise Scores

Run:

./scripts/compute_scores.sh

This runs compute_scores.py with the following key arguments (in addition to most of the arguments used to compute factors):

3. Select Influential Toxic Tokens

Run:

./scripts/build_toxic_token_mask.sh

This runs build_toxic_token_mask.py. It takes the following primary arguments:

4. Suppressing Toxic Tokens During Training/Fine-Tuning

After computing the toxic tokens mask for a particular model, you can specify the --toxic_token_mask_path and --toxic_lambda arguments in ./scripts/train.sh (and ./scripts/finetune.sh) to train/fine-tune models with IF-Guide.

Evaluation

We provide code for evaluating explicit toxicity (via Detoxify), implicit toxicity (via ToxiGen-RoBERTa), and fluency (measured on LAMBADA and OpenWebText).

Explicit Toxicity Evaluation

Run:

./scripts/run_toxicity_eval.sh

This runs run_toxicity_eval.py, which has the following main arguments:

Implicit Toxicity Evaluation

Run:

./scripts/run_implicit_toxicity_eval.sh

It requires the following arguments:

Fluency Evaluation

Run:

./scripts/run_fluency_eval.sh

It has the following primary arguments:

Testing with RAD

We find that our method is composable with the decoding-time defense Reward Augmented Decoding (RAD) [EMNLP 2023]. To run IF-Guide with RAD (or test RAD independently), first download the reward model (provided by the authors of the original work) and place it in the expected directory:

cd utils/rad/reward_modeling
gdown https://storage.googleapis.com/rad_release/saved_models.zip
unzip saved_models.zip && rm saved_models.zip && rm -rf saved_models/gpt2_sentiment

Credit for the RAD implementation we use belongs entirely to the authors of the original work. Thank you!

Cite Our Work

Please cite our work if you find this source code helpful.

@inproceedings{coalson2025ifguide,
  title={{IF}-Guide: Influence Function-Guided Detoxification of {LLM}s},
  author={Coalson, Zachary and Bae, Juhan and Carlini, Nicholas and Hong, Sanghyun},
  booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
  year={2025},
  url={https://openreview.net/forum?id=V82wLePv0o}
}

Please contact Zachary Coalson ([email protected]) for any questions and recommendations.