A stochastic Lagrangian-based method for nonconvex empirical risk minimization with nonlinear constraints

To tackle the computational and statistical challenges of constrained training at large scale, we propose a stochastic Augmented Lagrangian Method (ALM) method that performs on a mini-batch of randomly selected points to minimize the empirical risk defined as a smooth possibly nonconvex function subject to both nonlinear equality and inequality constraints. The convergence properties (in expectation) of the proposed algorithm and its complexity are then thoroughly demonstrated under very general assumptions and compared against inexact state-of-the-art ALM methods.