Information-theoretic principles in semantic and pragmatic communication

Maintaining useful semantic representations of the environment and pragmatically reasoning about utterances are crucial aspects of human language. However, it is not yet clear what computational principles could give rise to human-like semantics and pragmatics in machines. In this talk, I will propose a possible answer to this open question by hypothesizing that pressure for efficient coding may underlie both abilities. First, I will argue that languages efficiently encode meanings into words by optimizing the Information Bottleneck (IB) tradeoff between the complexity and accuracy of the lexicon. This proposal is supported by cross-linguistic data from three semantic domains: names for colors, artifacts, and animals. Furthermore, it suggests that semantic systems may evolve by navigating along the IB theoretical limit via an annealing-like process. This process generates quantitative predictions, which are directly supported by an analysis of recent data documenting changes over time in the color naming system of a single language. Second, I will derive a theoretical link between optimized semantic systems and local, context-dependent interactions that involve pragmatic skills. Specifically, I will show that pressure for efficient coding may also give rise to human pragmatic reasoning, as captured by the Rational Speech Act framework. This line of work identifies information-theoretic optimization principles that characterize human semantic and pragmatic communication, and that could be used to inform artificial agents with human-like communication systems.