CogALex 2.0: Impact of Data Quality on Lexical-Semantic Relation Prediction

Predicting lexical-semantic relations between word pairs has successfully been accomplished by pre-trained neural language models. An XLM-R-based approach, for instance, achieved the best performance differentiating between hypernymy, synonymy, antonymy, and random relations in four languages in the CogALex-VI 2020 shared task. However, the results also revealed strong performance divergences between languages and confusions of very specific relations, especially hypernymy and synonymy. Upon manual inspection a difference in data quality across languages and relations could be observed. We propose the improved dataset for lexical-semantic relation prediction and an evaluation and analysis of its impact across three pre-trained neural language models, including transfer learning.