Timezone: »

 
Spotlight
Optimal Query Complexities for Dynamic Trace Estimation
David Woodruff · Fred Zhang · Richard Zhang

Thu Dec 08 05:00 PM -- 07:00 PM (PST) @
in Lightning Talks 6B-1 »
We consider the problem of minimizing the number of matrix-vector queries needed for accurate trace estimation in the dynamic setting where our underlying matrix is changing slowly, such as during an optimization process. Specifically, for any $m$ matrices $\mathbf{A}_1,...,\mathbf{A}_m$ with consecutive differences bounded in Schatten-$1$ norm by $\alpha$, we provide a novel binary tree summation procedure that simultaneously estimates all $m$ traces up to $\epsilon$ error with $\delta$ failure probability with an optimal query complexity of $\widetilde{O}(m \alpha\sqrt{\log(1/\delta)}/\epsilon + m\log(1/\delta))$, improving the dependence on both $\alpha$ and $\delta$ from Dharangutte and Musco (NeurIPS, 2021). Our procedure works without additional norm bounds on $\mathbf{A}_i$ and can be generalized to a bound for the $p$-th Schatten norm for $p \in [1,2]$, giving a complexity of $\widetilde{O}(m \alpha(\sqrt{\log(1/\delta)}/\epsilon)^p +m \log(1/\delta))$. By using novel reductions to communication complexity and information-theoretic analyses of Gaussian matrices, we provide matching lower bounds for static and dynamic trace estimation in all relevant parameters, including the failure probability. Our lower bounds (1) give the first tight bounds for Hutchinson's estimator in the matrix-vector product model with Frobenius norm error {\it even in the static setting}, and (2) are the first unconditional lower bounds for dynamic trace estimation, resolving open questions of prior work.
Keywords: numerical linear algebra · trace estimation · query complexity lower bound

Author Information

David Woodruff (Carnegie Mellon University)
Fred Zhang (UC Berkeley)

I am a PhD student in the Theory Group of the EECS Department at UC Berkeley, advised by Jelani Nelson. My research lies broadly in algorithm design. I am particularly interested in questions arising from high-dimensional statistics, machine learning, and processing massive data.

Richard Zhang (Google Brain)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors