As inherently linguistic creatures, we tend to think of text as a simple domain: After all, there are only twenty-six letters in the English language, and basic tasks like keyword recognition and part-of-speech tagging have been routinely applied in industry for more than a decade. But words are not a domain in and of themselves. Rather, they function as abstract representations for other types of input, resulting in daunting levels of complexity. This presentation discusses some of the challenges presented by language tasks in general and by text-based games in particular, including partially observable state spaces, compositional and combinatorial action spaces, word-sense disambiguation, consumable rewards, and goal-directed inference.