Attacking Attention of Foundation Models Effectively Disrupts Downstream Tasks

Official PyTorch implementation of the paper "Attacking Attention of Foundation Models Effectively Disrupts Downstream Tasks", accepted in the Adversarial Machine Learning on Computer Vision: Foundation Models + X (ADVML) Workshop of CVPR 2025.

# Attacking Attention of Foundation Models Effectively Disrupts Downstream Tasks"
Hondamunige Prasanna Silva, Federico Becattini and Lorenzo Seidenari

Abstract: Foundation models represent the most prominent and recent paradigm shift in artificial intelligence. Foundation models are large models, trained on broad data that deliver high accuracy in many downstream tasks, often without finetuning. For this reason, models such as CLIP , DINO or Vision Transfomers (ViT) , are becoming the bedrock of many industrial AI-powered applications. However, the reliance on pre-trained foundation models also introduces significant security concerns, as these models are vulnerable to adversarial attacks. Such attacks involve deliberately crafted inputs designed to deceive AI systems,jeopardizing their reliability. This paper studies the vulnerabilities of vision foundation models, focusing specifically on CLIP and ViTs, and explores the transferability of adversarial attacks to downstream tasks. We introduce a novel attack, targeting the structure of transformer-based architectures in a task-agnostic fashion. We demonstrate the effectiveness of our attack on several downstream tasks: classification, captioning, image/text retrieval, segmentation and depth estimation

Table of Contents

• Abstract
• Installation
• Project Structure
• Download Models & Datasets
• How to Launch the Attack
• Usage
• Evaluation
• TODO
• Citation

Install conda env

conda env create -f environment.yml

Run the project as a Packages

pip install .  
pip instlal -e .  # If you want in edit mode!

Project Structure

attack-attention
├─ .gitignore
├─ .pre-commit-config.yaml
├─ LICENSE
├─ README.md
├─ checkpoint
│  ├─ albef
│  └─ tcl
├─ data
│  ├─ flickr30k-images
│  └─ val2014
├─ SGA
│  └─ ...
├─ data_annotation
│  ├─ coco_test.json
│  └─ flickr30k_test.json
├─ environment.yml
├─ eval_clip-vit2albef.py
├─ eval_clip-vit2clip-cnn.py
├─ eval_clip.py
├─ models
│  ├─ __init__.py
│  ├─ clip_model
│  │  ├─ model.py
│  │  └─ ..
│  └─ ..
└─ std_eval_idx
   ├─ flickr30k
   └─ mscoco

Download the model and datasets

Create checkpoint folder inside the root folder and download TCL and ALBEF models. The checkpoints of the fine-tuned VLP models is accessible in ALBEF, TCL.

Create data folder inside the root folder and download FLICKR30K, MSCOCO.

How to lunch SGA attack

Check readme.md inside SGA folder.

How to use

Parameters

Attack evaluation

To eval the attack without trasferability

python eval_clip.py  --config ./configs/Retrieval_coco.yaml \
   --source_model ViT-B/16 --original_rank_index ./std_eval_idx/mscoco/ \
   --loss atn --method 1;

To eval the attack with trasferability (ALBEF)

python eval_clip-vit2albef.py  --config ./configs/Retrieval_flickr.yaml \
   --source_model ViT-B/16  --target_model ALBEF --target_ckpt ./checkpoint/albef/flickr30k.pth \
   --original_rank_index ./std_eval_idx/flickr30k/ --loss atn --method 1

To eval the attack with trasferability (TCL)

python eval_clip-vit2albef.py --config ./configs/Retrieval_flickr.yaml \
   --source_model ViT-B/16 --target_model TCL --target_ckpt ./checkpoint/tcl/checkpoint_flickr_finetune.pth \
   --original_rank_index ./std_eval_idx/flickr30k/ --loss atn --method 1

To eval the attack with trasferability (CLIP-CNN)

python eval_clip-vit2clip-cnn.py --config ./configs/Retrieval_flickr.yaml \
--source_model ViT-B/16  --target_model RN101 \
--original_rank_index ./std_eval_idx/flickr30k/ --loss atn --method 1

Parameter used for experiments

--source_model ViT-B/16 --method 3 --loss emb

TODO

• Complete the repo with all the downstream task used during experiments
• Refactor the code

Citation

If you find this work useful, please consider citing it:

@InProceedings{Silva_2025_CVPR,
    author    = {Silva, Hondamunige Prasanna and Becattini, Federico and Seidenari, Lorenzo},
    title     = {Attacking Attention of Foundation Models Disrupts Downstream Tasks},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops},
    month     = {June},
    year      = {2025},
    pages     = {3534-3543}
}```