Timezone: »

 
Poster
Optimal Iterative Sketching Methods with the Subsampled Randomized Hadamard Transform
Jonathan Lacotte · Sifan Liu · Edgar Dobriban · Mert Pilanci

Wed Dec 09 09:00 PM -- 11:00 PM (PST) @ Poster Session 4 #1246
in Poster Session 5 »

Random projections or sketching are widely used in many algorithmic and learning contexts. Here we study the performance of iterative Hessian sketch for least-squares problems. By leveraging and extending recent results from random matrix theory on the limiting spectrum of matrices randomly projected with the subsampled randomized Hadamard transform, and truncated Haar matrices, we can study and compare the resulting algorithms to a level of precision that has not been possible before. Our technical contributions include a novel formula for the second moment of the inverse of projected matrices. We also find simple closed-form expressions for asymptotically optimal step-sizes and convergence rates. These show that the convergence rate for Haar and randomized Hadamard matrices are identical, and asymptotically improve upon Gaussian random projections. These techniques may be applied to other algorithms that employ randomized dimension reduction.

Keywords: Deep Learning · Algorithms -> Classification; Applications -> Activity and Event Recognition; Applications -> Computer Vision; Applications

Author Information

Jonathan Lacotte (Stanford University)
Sifan Liu (Stanford University)
Edgar Dobriban (University of Pennsylvania)
Mert Pilanci (Stanford)

More from the Same Authors