Vision Toolkit Based on Hierarchical Temporal Memory

We demonstrate a Vision Toolkit based on Hierarchical Temporal Memory (HTM). Hierarchical Temporal Memory is a technology being developed by Numenta based on a theory of operation of the neocortex. HTMs model data by learning spatial co-occurrences and temporal sequences at multiple levels in a hierarchy. More details on the technology can be obtained from http://www.numenta.com/for-developers/education/general-overview-htm.php The vision toolkit has capabilities for object recognition, top-down attention and shape similarity search. The vision toolkit demonstrates robust invariant recognition of gray-scale objects in the presence of noise and large amounts of clutter. The HTM network behind this learns its invariant representation at every level through exposure to video sequences of moving objects. Inference in the network is done using Bayesian belief propagation. We demonstrate a network that handles 25 categories of images with large intra category variations. In addition to object recognition, the toolkit can also do visual similarity search. Traditionally, multiple objects in a scene are recognized using scan windows of different sizes. The HTM vision network can recognize and localize multiple objects in a scene using a top-down attention mechanism. The feed-forward pass on a scene gives a set of hypotheses about the objects present in the scene. The network localizes objects through the feedback propagation of the top hypotheses. One important aspect of the vision toolkit is the ease of use. The vision toolkit is a self-contained system that enables a user to create a recognition network with very little programming. The vision toolkit is built on top of NuPIC - Numenta Platform for intelligent computing. NuPIC allows for rapid prototyping using hierarchical networks. It uses python as the scripting language with math libraries written in C++ for faster execution. Parallelization capabilities are built in. We also demonstrate the generic capabilities of NuPIC in learning higher-order spatio-temporal models of data. We demonstrate unsupervised learning of action models using motion-capture data and hierarchical sequence prediction using web-navigation data.