Bergson: An Open Source Library for Data Attribution

Data attribution is a promising field in interpretability that aims to explain model behavior through the influence of its training data, with applications including debugging undesirable model behavior and training dataset curation. However, significant engineering effort is required to perform it at scale, and many cutting-edge techniques lack open-source tooling and support. Bergson is an open-source library that closes this gap, providing a host of techniques that scale to very large language models and pre-training datasets. The library natively supports on-disk gradient stores and multi-node distributed training, and provides quality-of-life tools for researchers. Finally, Bergson includes the first open-source implementations of three leading data attribution methods: MAGIC, SOURCE, and TrackStar. The library is available at https://anonymous.4open.science/r/bergson-BD34.