Invited Talk: Evaluating the Ethical Competence of LLMs

Existing approaches to evaluating LLM ethical competence place too much emphasis on the verdicts—of permissibility and impermissibility—that they render. But ethical competence doesn’t consist in one’s judgments conforming to those of a cohort of crowdworkers. It consists in being able to identify morally relevant features, prioritise among them, associate them with reasons and weave them into a justified conclusion. We identify the limitations of existing evals for ethical competence, provide an account of moral reasoning that can ground better alternatives, and discuss the practical—and philosophical—implications if LLMs ultimately do prove to be adept moral reasoners.