|
Workshop
|
|
MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
Artavazd Maranjyan · Omar Shaikh Omar · Peter Richtarik
|
|
|
Workshop
|
|
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
Frederik Kunstner · Robin Yadav · Alan Milligan · Mark Schmidt · Alberto Bietti
|
|
|
Workshop
|
|
On Convergence of SGD with Adaptive Clipping
Egor Shulgin · Peter Richtarik
|
|
|
Workshop
|
Sun 11:30
|
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning, Misha Belkin
Misha Belkin
|
|
|
Workshop
|
|
dSTAR: Straggler Tolerant and Byzantine Resilient Distributed SGD
Jiahe Yan · Pratik Chaudhari · Leonard Kleinrock
|
|
|
Workshop
|
|
Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training
Anchit Jain · Rozhin Nobahari · Aristide Baratin · Stefano Sarao Mannelli
|
|
|
Workshop
|
Sun 14:30
|
Talk 1: *MindFlayer: Efficient Asynchronous Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times* and Talk 2: *Provable non-accelerations of the heavy-ball method*
Artavazd Maranjyan · Gauthier Gidel
|
|
|
Workshop
|
|
Connections between Schedule-Free SGD, Accelerated SGD Variants, and Weight Averaging
Depen Morwani · Nikhil Vyas · Hanlin Zhang · Sham Kakade
|
|
|
Workshop
|
|
Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training
Hiroki Naganuma · Xinzhi Zhang · Man-Chung Yue · Ioannis Mitliagkas · Russell J. Hewett · Philipp Witte · Yin Tat Lee
|
|
|
Poster
|
Wed 11:00
|
Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD
Aniket Das · Dheeraj Nagaraj · Soumyabrata Pal · Arun Suggala · Prateek Varshney
|
|
|
Workshop
|
|
From Gradient Clipping to Normalization for Heavy Tailed SGD
Florian Hübler · Ilyas Fatkhullin · Niao He
|
|
|
Poster
|
Thu 11:00
|
Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
Alexander Tyurin · Marta Pozzi · Ivan Ilin · Peter Richtarik
|
|
