Skip to yearly menu bar Skip to main content


Workshop

Red Teaming GenAI: What Can We Learn from Adversaries?

Valeriia Cherepanova ⋅ Bo Li ⋅ Niv Cohen ⋅ Yifei Wang ⋅ Yisen Wang ⋅ Avital Shafran ⋅ Nil-Jana Akpinar ⋅ James Zou
[ OpenReview] 

Abstract

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles
9:00 AM
Coffee break
9:30 AM
Opening Remark
10:45 AM
Contributed Talk 1: iART - Imitation guided Automated Red Teaming
Sajad Mousavi ⋅ Desik Rengarajan ⋅ Ashwin Ramesh Babu ⋅ Vineet Gundecha ⋅ Avisek Naug ⋅ Sahand Ghorbanpour ⋅ Ricardo Luna Gutierrez ⋅ Antonio Guillen-Perez ⋅ Paolo Faraboschi ⋅ Soumyendu Sarkar
10:55 AM
Contributed Talk 2: Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
Rylan Schaeffer ⋅ Dan Valentine ⋅ Luke Bailey ⋅ James Chua ⋅ Zane Durante ⋅ Cristobal Eyzaguirre ⋅ Joe Benton ⋅ Brando Miranda ⋅ Henry Sleight ⋅ Tony Wang ⋅ John Hughes ⋅ Rajashree Agrawal ⋅ Mrinank Sharma ⋅ Scott Emmons ⋅ Sanmi Koyejo ⋅ Ethan Perez
11:05 AM
Contributed Talk 3: LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet
Nathaniel Li ⋅ Ziwen Han ⋅ Ian Steneker ⋅ Willow Primack ⋅ Riley Goodside ⋅ Hugh Zhang ⋅ Zifan Wang ⋅ Cristina Menghini ⋅ Summer Yue
11:20 AM
Panel Discussion
Roei Schuster ⋅ Yaron Singer ⋅ Alex Tamkin ⋅ Bo Li
12:00 PM
Lunch
12:00 PM
Poster Session
3:00 PM
Coffee Break
3:30 PM
Invited talk 5: Vitaly Shmatikov and Q&A
Vitaly Shmatikov
4:15 PM
Invited talk 6: Gowthami Somepalli and Q&A
Gowthami Somepalli
4:30 PM
Contributed Talk 4: Rethinking LLM Memorization through the Lens of Adversarial Compression
Avi Schwarzschild ⋅ Zhili Feng ⋅ Pratyush Maini ⋅ Zachary Lipton ⋅ J. Zico Kolter
4:40 PM
Contributed Talk 5: A Realistic Threat Model for Large Language Model Jailbreaks
Valentyn Boreiko ⋅ Alexander Panfilov ⋅ Vaclav Voracek ⋅ Matthias Hein ⋅ Jonas Geiping
4:50 PM
Contributed Talk 6: Infecting LLM Agents via Generalizable Adversarial Attack
Weichen Yu ⋅ Kai Hu ⋅ Tianyu Pang ⋅ Chao Du ⋅ Min Lin ⋅ Matt Fredrikson
5:20 PM
Closing Remarks
Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI
Ambrish Rawat ⋅ Stefan Schoepf ⋅ Giulio Zizzo ⋅ Giandomenico Cornacchia ⋅ Muhammad Zaid Hameed ⋅ Kieran Fraser ⋅ Erik Miehling ⋅ Beat Buesser ⋅ Elizabeth Daly ⋅ Mark Purcell ⋅ Prasanna Sattigeri ⋅ Pin-Yu Chen ⋅ Kush Varshney
Infecting LLM Agents via Generalizable Adversarial Attack
Weichen Yu ⋅ Kai Hu ⋅ Tianyu Pang ⋅ Chao Du ⋅ Min Lin ⋅ Matt Fredrikson
Decoding Biases: An Analysis of Automated Methods and Metrics for Gender Bias Detection in Language Models
Shachi H. Kumar ⋅ Saurav Sahay ⋅ Sahisnu Mazumder ⋅ Eda Okur ⋅ Ramesh Manuvinakurike ⋅ Nicole Beckage ⋅ Hsuan Su ⋅ Hung-yi Lee ⋅ Lama Nachman
Interactive Semantic Interventions for VLMs: Breaking VLMs with Human Ingenuity
Lukas Klein ⋅ Kenza Amara ⋅ Carsten Lüth ⋅ Hendrik Strobelt ⋅ Mennatallah El-Assady ⋅ Paul Jaeger
Rethinking LLM Memorization through the Lens of Adversarial Compression
Avi Schwarzschild ⋅ Zhili Feng ⋅ Pratyush Maini ⋅ Zachary Lipton ⋅ J. Zico Kolter
Learning Diverse Attacks on Large Language Models for Robust Red-Teaming and Safety Tuning
Seanie Lee ⋅ Minsu Kim ⋅ Lynn Cherif ⋅ David Dobre ⋅ Juho Lee ⋅ Sung Ju Hwang ⋅ Kenji Kawaguchi ⋅ Gauthier Gidel ⋅ Yoshua Bengio ⋅ Nikolay Malkin ⋅ Moksh Jain
SAGE-RT: Synthetic Alignment data Generation for Safety Evaluation and Red Teaming
Anurakt Kumar ⋅ Divyanshu Kumar ⋅ Jatan Loya ⋅ Nitin Aravind Birur ⋅ Tanay Baswa ⋅ Sahil Agarwal ⋅ Prashanth Harshangi
An Adversarial Perspective on Machine Unlearning for AI Safety
Jakub Łucki ⋅ Boyi Wei ⋅ Yangsibo Huang ⋅ Peter Henderson ⋅ Florian Tramer ⋅ Javier Rando
Stability Evaluation of Large Language Models via Distributional Perturbation Analysis
Jiashuo Liu ⋅ Jiajin Li ⋅ Peng Cui ⋅ Jose Blanchet
Lessons From Red Teaming 100 Generative AI Products
Blake Bullwinkel ⋅ Amanda Minnich ⋅ Shiven Chawla ⋅ Gary Lopez Munoz ⋅ Martin Pouliot ⋅ Whitney Maxwell ⋅ Joris de Gruyter ⋅ Katherine Pratt ⋅ Saphir Qi ⋅ Nina Chikanov ⋅ Roman Lutz ⋅ Raja Sekhar Rao Dheekonda ⋅ Bolor-Erdene Jagdagdorj ⋅ Rich Lundeen ⋅ Sam Vaughan ⋅ Victoria Westerhoff ⋅ Pete Bryan ⋅ Ram Shankar Siva Kumar ⋅ Yonatan Zunger ⋅ Mark Russinovich
LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet
Nathaniel Li ⋅ Ziwen Han ⋅ Ian Steneker ⋅ Willow Primack ⋅ Riley Goodside ⋅ Hugh Zhang ⋅ Zifan Wang ⋅ Cristina Menghini ⋅ Summer Yue
Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage
Rafi Rashid ⋅ Jing Liu ⋅ Toshiaki Koike-Akino ⋅ Shagufta Mehnaz ⋅ Ye Wang
Steganography in Large Language Models: Investigating Emergence and Mitigations
Yohan Mathew ⋅ Robert McCarthy ⋅ Ollie Matthews ⋅ Joan Velja ⋅ Nandi Schoots ⋅ Dylan Cope
A Realistic Threat Model for Large Language Model Jailbreaks
Valentyn Boreiko ⋅ Alexander Panfilov ⋅ Vaclav Voracek ⋅ Matthias Hein ⋅ Jonas Geiping
Decompose, Recompose, and Conquer: Multi-modal LLMs are Vulnerable to Compositional Adversarial Attacks in Multi-Image Queries
Julius Broomfield ⋅ George Ingebretsen ⋅ Reihaneh Iranmanesh ⋅ Sara Pieri ⋅ Ethan Kosak-Hine ⋅ Tom Gibbs ⋅ Reihaneh Rabbany ⋅ Kellin Pelrine
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
Rylan Schaeffer ⋅ Dan Valentine ⋅ Luke Bailey ⋅ James Chua ⋅ Zane Durante ⋅ Cristobal Eyzaguirre ⋅ Joe Benton ⋅ Brando Miranda ⋅ Henry Sleight ⋅ Tony Wang ⋅ John Hughes ⋅ Rajashree Agrawal ⋅ Mrinank Sharma ⋅ Scott Emmons ⋅ Sanmi Koyejo ⋅ Ethan Perez
SkewAct: Red Teaming Large Language Models via Activation-Skewed Adversarial Prompt Optimization
Hanxi Guo ⋅ Siyuan Cheng ⋅ Guanhong Tao ⋅ Guangyu Shen ⋅ Zhuo Zhang ⋅ Shengwei An ⋅ Kaiyuan Zhang ⋅ Xiangyu Zhang
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs
Aly Kassem ⋅ Omar Mahmoud ⋅ Niloofar Mireshghallah ⋅ Hyunwoo Kim ⋅ Yulia Tsvetkov ⋅ Yejin Choi ⋅ Sherif Saad ⋅ Santu Rana
TOFU: A Task of Fictitious Unlearning for LLMs
Pratyush Maini ⋅ Zhili Feng ⋅ Avi Schwarzschild ⋅ Zachary Lipton ⋅ J. Zico Kolter
iART - Imitation guided Automated Red Teaming
Sajad Mousavi ⋅ Desik Rengarajan ⋅ Ashwin Ramesh Babu ⋅ Vineet Gundecha ⋅ Avisek Naug ⋅ Sahand Ghorbanpour ⋅ Ricardo Luna Gutierrez ⋅ Antonio Guillen-Perez ⋅ Paolo Faraboschi ⋅ Soumyendu Sarkar
Does Refusal Training in LLMs Generalize to the Past Tense?
Maksym Andriushchenko ⋅ Nicolas Flammarion
Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models
Hongfu Liu ⋅ Yuxi Xie ⋅ Ye Wang ⋅ Michael Qizhe Shieh
CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation
Tong Chen ⋅ Akari Asai ⋅ Niloofar Mireshghallah ⋅ Sewon Min ⋅ James Grimmelmann ⋅ Yejin Choi ⋅ Hannaneh Hajishirzi ⋅ Luke Zettlemoyer ⋅ Pang Wei Koh
Curiosity-driven Red teaming for Large Language Models
Zhang-Wei Hong ⋅ Idan Shenfeld ⋅ Tsun-Hsuan Johnson Wang ⋅ Yung-Sung Chuang ⋅ Aldo Pareja ⋅ Jim Glass ⋅ Akash Srivastava ⋅ Pulkit Agrawal
Adversarial Negotiation Dynamics in Generative Language Models
Arinbjörn Kolbeinsson ⋅ Benedikt Kolbeinsson
Algorithmic Oversight for Deceptive Reasoning
Ege Onur Taga ⋅ Mingchen Li ⋅ Yongqi Chen ⋅ Samet Oymak