Getting Started with Model Cards

Tech companies are proficient at creating and maintaining product specifications. With machine learning applications being increasingly deployed in the real world, it is important to be transparent about scope and limitations. A straightforward way to do that is through model cards [1]. Model card is a short document that records model behavior, context, intended use, and details about the data.

The goal of our cross-functional effort is to help make it easy to create model cards. We share our experience with creating model cards for two kinds of machine learning models: medical image classification [3, 4] and credit default prediction [5]. We describe experiments we performed, the tools we used, and datasets we analyzed [3, 4, 5]. We also propose a model card template created using a user experience design tool.

We describe some of the challenges we encountered, to help make it visible what it would take for wide adoption of model cards by practitioners. We ask and provide some answers to questions like: What data will you need? How time- and resource-intensive will the experiments be? How to interpret and communicate the experiment results?