Timezone: »

 
Poster
Maximum Covariance Unfolding : Manifold Learning for Bimodal Data
Vijay Mahadevan · Chi Wah Wong · Jose Costa Pereira · Tom Liu · Nuno Vasconcelos · Lawrence Saul

Wed Dec 14 08:45 AM -- 02:59 PM (PST) @
in Poster Session »

We propose maximum covariance unfolding (MCU), a manifold learning algorithm for simultaneous dimensionality reduction of data from different input modalities. Given high dimensional inputs from two different but naturally aligned sources, MCU computes a common low dimensional embedding that maximizes the cross-modal (inter-source) correlations while preserving the local (intra-source) distances. In this paper, we explore two applications of MCU. First we use MCU to analyze EEG-fMRI data, where an important goal is to visualize the fMRI voxels that are most strongly correlated with changes in EEG traces. To perform this visualization, we augment MCU with an additional step for metric learning in the high dimensional voxel space. Second, we use MCU to perform cross-modal retrieval of matched image and text samples from Wikipedia. To manage large applications of MCU, we develop a fast implementation based on ideas from spectral graph theory. These ideas transform the original problem for MCU, one of semidefinite programming, into a simpler problem in semidefinite quadratic linear programming.

[ PDF] 

Author Information

Vijay Mahadevan (Yahoo!)
Chi Wah Wong (University of California San Diego)
Jose Costa Pereira (UC San Diego)
Tom Liu (University of California, San Diego)
Nuno Vasconcelos (UC San Diego)
Lawrence Saul (Flatiron Institute)

More from the Same Authors