Spotlight
in
Workshop: Information-Theoretic Principles in Cognitive Systems (InfoCog)
Natural Language Systematicity from a Constraint on Excess Entropy
Richard Futrell
[
Abstract
]
[ Project Page ]
presentation:
Information-Theoretic Principles in Cognitive Systems (InfoCog)
Fri 15 Dec 6:15 a.m. PST — 3:30 p.m. PST
[
OpenReview]
Fri 15 Dec 9:46 a.m. PST
— 9:49 a.m. PST
Fri 15 Dec 6:15 a.m. PST — 3:30 p.m. PST
Abstract:
Natural language is systematic: utterances are composed of individually meaningful parts which are typically concatenated together. We argue that natural-language-like systematicity arises in codes when they are constrained by excess entropy, the mutual information between the past and the future of a process. In three examples, we show that codes with natural-language-like systematicity have lower excess entropy than matched alternatives.
Chat is not available.