Languages You Know Influence Those You Learn: Impact of Language Characteristics on Multi-Lingual Text-to-Text Transfer

In this work, we analyze a pre-trained mT5 \cite{xue2020mt5} to discover the attributes of cross-lingual connections learned by this model.Through a statistical interpretation framework over 90 language pairs across three tasks, we show that transfer performance can be significantly modeled by a few linguistic and data-derived features.These observations enable us to interpret cross-lingual understanding of the mT5 model.Through these observations, one can favorably choose the best source language for a task, and can anticipate its training data demands.A key finding of this work is that similarity of syntax, morphology and phonology are good predictors of cross-lingual transfer, significantly more than just the lexical similarity of languages.For a given language, we are able to predict zero-shot performance, that increases on a logarithmic scale with the number of few-shot target language data points.