Evaluating Refusal

How might we find a place for refusal within the evaluation of Generative AI systems? Current evaluation frameworks justifiably focus on possible uses of models. Given the myriad unsolved issues in Generative AI systems and their rapid rise, some developers and potential users are rejecting their use in both public and private settings (Solaiman et al . (2024)). Respecting the autonomy of users means respecting their decision not to use these technologies. Based on literature on refusal and data ethics, we provide several provocations positing that refusal is a generative act, and advocating the inclusion of refusal in evaluation frameworks.