Timezone: »

Sensemaking Interfaces for Human Evaluation of Language Model Outputs
Katy Gero · Jonathan Kummerfeld · Elena Glassman

Sat Dec 03 01:45 PM -- 01:55 PM (PST) @
Event URL: https://openreview.net/forum?id=ECtI2w8RMUf »

Ensuring a language model doesn't generate problematic text is difficult. Traditional evaluation methods, like automatic measures or human annotation, can fail to detect all problems, whether because system designers were not aware of a kind of problem they should attempt to detect, or because an automatic measure fails to reliably detect certain kinds of problems. In this paper we propose sensemaking tools as a robust and open-ended method to evaluate the large number of linguistic outputs produced by a language model. We demonstrate one potential sensemaking interface based on concordance tables, showing that we are able to detect problematic outputs and distributional shifts in minutes, despite not knowing exactly what kind of problems to look for.

Author Information

Katy Gero (Columbia University)
Jonathan Kummerfeld (University of Sydney)
Jonathan Kummerfeld

Jonathan K. Kummerfeld is a Senior Lecturer (ie., research tenure-track Assistant Professor) in the School of Computer Science at the University of Sydney. He completed his Ph.D. at the University of California, Berkeley, advised by Prof. Dan Klein, and was a postdoc at the University of Michigan from 2016 - 2021, working with a range of AI faculty, including Rada Mihalcea, Dragomir Radev, and Satinder Singh. Jonathan’s research has revealed new challenges in syntactic parsing, coreference resolution, and dialogue. He has proposed models and algorithms to address these challenges, improving the speed and accuracy of natural language processing systems. He has been on the program committee for over 50 conferences and workshops. He currently serves as the Co-CTO of ACL Rolling Review, and is a standing reviewer for the Computational Linguistics journal and the Transactions of the Association for Computational Linguistics journal.

Elena Glassman (Harvard University)

I design, build and evaluate systems for comprehending and interacting with population-level structure and trends in large code and data corpora. I am an Assistant Professor of Computer Science at the Harvard Paulson School of Engineering & Applied Sciences and the Stanley A. Marks & William H. Marks Professor at the Radcliffe Institute for Advanced Study, specializing in human-computer interaction. At MIT, I earned a PhD and MEng in Electrical Engineering and Computer Science and a BS in Electrical Science and Engineering. Before joining Harvard, I was a postdoctoral scholar in Electrical Engineering and Computer Science at the University of California, Berkeley, where I received the Berkeley Institute for Data Science Moore/Sloan Data Science Fellowship.

More from the Same Authors