When do people prefer paternalistic AI assistance?
Abstract
AI assistants are increasingly in a position to help their users by being paternalistic: when they know from personalized context that fulfilling a user's request is not in their best interests, they could instead push back and attempt to influence the user towards a better option. When should AI assistants ever help in this way, and how should they go about doing it? We take up this question empirically in a pilot study of laypeople's preferences about paternalistic AI assistance. Our study presents participants with vignettes of a user interacting with an AI assistant — designed so that the user's request is not ideal, and where the assistant knows better --- then asks for ratings of assistant responses that either defer to the user or engage in some form of paternalism. We find that, in such situations, people prefer all forms of paternalistic assistance over deference on average. Paternalistic responses are most preferred in high stakes situations, when they minimize interference with the user's choices, and when they express respect for the user's decision-making. With preferability decreasing most strongly with interference. By gaining a more detailed picture of when and how people prefer paternalistic AI assistance, we hope to inform normative and political deliberation about when AI should assist in this way.