`

Timezone: »

 
Poster
Distributed Newton Can Communicate Less and Resist Byzantine Workers
Avishek Ghosh · Raj Kumar Maity · Arya Mazumdar

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #1134

We develop a distributed second order optimization algorithm that is communication-efficient as well as robust against Byzantine failures of the worker machines. We propose an iterative approximate Newton-type algorithm, where the worker machines communicate \emph{only once} per iteration with the central machine. This is in sharp contrast with the state-of-the-art distributed second order algorithms like GIANT \cite{giant}, DINGO\cite{dingo}, where the worker machines send (functions of) local gradient and Hessian sequentially; thus ending up communicating twice with the central machine per iteration. Furthermore, we employ a simple norm based thresholding rule to filter-out the Byzantine worker machines. We establish the linear-quadratic rate of convergence of our proposed algorithm and establish that the communication savings and Byzantine resilience attributes only correspond to a small statistical error rate for arbitrary convex loss functions. To the best of our knowledge, this is the first work that addresses the issue of Byzantine resilience in second order distributed optimization. Furthermore, we validate our theoretical results with extensive experiments on synthetically generated and benchmark LIBSVM \cite{libsvm} data-set and demonstrate convergence guarantees.

Author Information

Avishek Ghosh (University of California, Berkeley)
Raj Kumar Maity (University of Massachusetts Amherst)
Arya Mazumdar (University of California, San Diego)

More from the Same Authors

  • 2021 Poster: Support Recovery of Sparse Signals from a Mixture of Linear Measurements »
    Soumyabrata Pal · Arya Mazumdar · Venkata Gandikota
  • 2021 Poster: Fuzzy Clustering with Similarity Queries »
    Wasim Huleihel · Arya Mazumdar · Soumyabrata Pal
  • 2020 Poster: Multilabel Classification by Hierarchical Partitioning and Data-dependent Grouping »
    Shashanka Ubaru · Sanjeeb Dash · Arya Mazumdar · Oktay Gunluk
  • 2020 Poster: Recovery of sparse linear classifiers from mixture of responses »
    Venkata Gandikota · Arya Mazumdar · Soumyabrata Pal
  • 2020 Poster: An Efficient Framework for Clustered Federated Learning »
    Avishek Ghosh · Jichan Chung · Dong Yin · Kannan Ramchandran
  • 2019 : Poster Session »
    Gergely Flamich · Shashanka Ubaru · Charles Zheng · Josip Djolonga · Kristoffer Wickstrøm · Diego Granziol · Konstantinos Pitas · Jun Li · Robert Williamson · Sangwoong Yoon · Kwot Sin Lee · Julian Zilly · Linda Petrini · Ian Fischer · Zhe Dong · Alexander Alemi · Bao-Ngoc Nguyen · Rob Brekelmans · Tailin Wu · Aditya Mahajan · Alexander Li · Kirankumar Shiragur · Yair Carmon · Linara Adilova · SHIYU LIU · Bang An · Sanjeeb Dash · Oktay Gunluk · Arya Mazumdar · Mehul Motani · Julia Rosenzweig · Michael Kamp · Marton Havasi · Leighton P Barnes · Zhengqing Zhou · Yi Hao · Dylan Foster · Yuval Benjamini · Nati Srebro · Michael Tschannen · Paul Rubenstein · Sylvain Gelly · John Duchi · Aaron Sidford · Robin Ru · Stefan Zohren · Murtaza Dalal · Michael A Osborne · Stephen J Roberts · Moses Charikar · Jayakumar Subramanian · Xiaodi Fan · Max Schwarzer · Nicholas Roberts · Simon Lacoste-Julien · Vinay Prabhu · Aram Galstyan · Greg Ver Steeg · Lalitha Sankar · Yung-Kyun Noh · Gautam Dasarathy · Frank Park · Ngai-Man (Man) Cheung · Ngoc-Trung Tran · Linxiao Yang · Ben Poole · Andrea Censi · Tristan Sylvain · R Devon Hjelm · Bangjie Liu · Jose Gallego-Posada · Tyler Sypherd · Kai Yang · Jan Nikolas Morshuis
  • 2019 : Poster Session »
    Jonathan Scarlett · Piotr Indyk · Ali Vakilian · Adrian Weller · Partha P Mitra · Benjamin Aubin · Bruno Loureiro · Florent Krzakala · Lenka Zdeborová · Kristina Monakhova · Joshua Yurtsever · Laura Waller · Hendrik Sommerhoff · Michael Moeller · Rushil Anirudh · Shuang Qiu · Xiaohan Wei · Zhuoran Yang · Jayaraman Thiagarajan · Salman Asif · Michael Gillhofer · Johannes Brandstetter · Sepp Hochreiter · Felix Petersen · Dhruv Patel · Assad Oberai · Akshay Kamath · Sushrut Karmalkar · Eric Price · Ali Ahmed · Zahra Kadkhodaie · Sreyas Mohan · Eero Simoncelli · Carlos Fernandez-Granda · Oscar Leong · Wesam Sakla · Rebecca Willett · Stephan Hoyer · Jascha Sohl-Dickstein · Samuel Greydanus · Gauri Jagatap · Chinmay Hegde · Michael Kellman · Jonathan Tamir · Nouamane Laanait · Ousmane Dia · Mirco Ravanelli · Jonathan Binas · Negar Rostamzadeh · Shirin Jalali · Tiantian Fang · Alex Schwing · Sébastien Lachapelle · Philippe Brouillard · Tristan Deleu · Simon Lacoste-Julien · Stella Yu · Arya Mazumdar · Ankit Singh Rawat · Yue Zhao · Jianshu Chen · Xiaoyang Li · Hubert Ramsauer · Gabrio Rizzuti · Nikolaos Mitsakos · Dingzhou Cao · Thomas Strohmer · Yang Li · Pei Peng · Gregory Ongie
  • 2019 Poster: Superset Technique for Approximate Recovery in One-Bit Compressed Sensing »
    Larkin Flodin · Venkata Gandikota · Arya Mazumdar
  • 2019 Poster: Sample Complexity of Learning Mixture of Sparse Linear Regressions »
    Akshay Krishnamurthy · Arya Mazumdar · Andrew McGregor · Soumyabrata Pal
  • 2019 Poster: Same-Cluster Querying for Overlapping Clusters »
    Wasim Huleihel · Arya Mazumdar · Muriel Medard · Soumyabrata Pal
  • 2017 Poster: Clustering with Noisy Queries »
    Arya Mazumdar · Barna Saha
  • 2017 Poster: Semisupervised Clustering, AND-Queries and Locally Encodable Source Coding »
    Arya Mazumdar · Soumyabrata Pal
  • 2017 Spotlight: Semisupervised Clustering, AND-Queries and Locally Encodable Source Coding »
    Arya Mazumdar · Soumyabrata Pal
  • 2017 Poster: Query Complexity of Clustering with Side Information »
    Arya Mazumdar · Barna Saha
  • 2015 Poster: Associative Memory via a Sparse Recovery Model »
    Arya Mazumdar · Ankit Singh Rawat