Skip to yearly menu bar Skip to main content


Poster

Improvements to the Sequence Memoizer

Jan Gasthaus · Yee Whye Teh


Abstract:

The sequence memoizer is a model for sequence data with state-of-the-art performance on language modeling and compression. We propose a number of improvements to the model and inference algorithm, including an enlarged range of hyperparameters, a memory-efficient representation, and inference algorithms operating on the new representation. Our derivations are based on precise definitions of the various processes that will also allow us to provide an elementary proof of the "mysterious" coagulation and fragmentation properties used in the original paper on the sequence memoizer by Wood et al. (2009). We present some experimental results supporting our improvements.

Live content is unavailable. Log in and register to view live content