Transparency as an Antidote to Self-Preferencing Biases in LLM-Aided Hiring

In recent times, LLMs have become an integral part of hiring pipelines, with firms using them regularly to automate steps like resume evaluation and screening. At the same time, LLMs have also found heavy use among job-seekers who use them to prepare resumes and cover-letters tailored for various positions. Despite the demonstrated benefits of AI, it has been empirically observed across a variety of domains that in the context of evaluation tasks, the use of LLMs can lead to the emergence of a new form of interaction bias called ``self-preferencing". Specifically in the context of hiring, evaluator LLMs tend to prefer writing samples from models of the same category, thus favoring specific sub-groups of applicants (who use similar models as the firm) and distorting hiring outcomes. In this work, we investigate how firms can mitigate the effects of such biases and make accurate decisions when they are using i) a single LLM or ii) multiple LLMs in an aggregated way to generate estimates of applicant quality. When applicants are non-strategic in their LLM usage and lack clarity about the firm's evaluation mechanism, we find that it is not possible to eliminate disparities entirely and they continue to persist. However, transparency on the firm-side can have a powerful impact on outcomes --- when applicants know how the firm is using LLMs in their evaluation process, it allows them to strategize and induces equilibria where the disparities vanish and all applicants are treated fairly. Our work thus highlights the positive role of transparency in enabling safer integration of AI tools into high-stakes decision pipelines.