Convergence of the Generalized Alternating Direction Method of Multipliers for a Class of Nonconvex Optimization Problems

WANG Xin; GUO Ke

doi:10.21656/1000-0887.380334

Volume 39 Issue 12

Turn off MathJax

Article Contents

Article Navigation > Applied Mathematics and Mechanics > 2018 > 39(12): 1410-1425

WANG Xin, GUO Ke. Convergence of the Generalized Alternating Direction Method of Multipliers for a Class of Nonconvex Optimization Problems[J]. Applied Mathematics and Mechanics, 2018, 39(12): 1410-1425. doi: 10.21656/1000-0887.380334

Citation:

PDF( 432 KB)

Convergence of the Generalized Alternating Direction Method of Multipliers for a Class of Nonconvex Optimization Problems

doi: 10.21656/1000-0887.380334

WANG Xin,
GUO Ke

School of Mathematics and Information, China West Normal University,Nanchong, Sichuan 637002, P.R.China

Funds: The National Natural Science Foundation of China(11571178; 11801455)

Received Date: 2017-12-27
Rev Recd Date: 2018-10-18
Publish Date: 2018-12-01

Abstract

Abstract

The generalized alternating direction method of multipliers (GADMM) for the minimization problems of the sum of 2 functions with linear constraints was considered, where one function was convex and the other can be expressed as the difference of 2 convex functions. For each subproblem in the GADMM, the linearization technique in the convex function difference algorithm was employed. Under the assumptions that the associated functions satisfy the Kurdyka-ojasiewicz inequality, the sequence generated with the GADMM converges to a critical point of the augmented Lagrangian function, while the penalty parameter in the augmented Lagrangian function is sufficiently large. At last, the convergence rate of the algorithm was established.
- generalized alternating direction method of multipliers,
- Kurdyka-ojasiewicz inequality,
- nonconvex optimization,
- convergence

FullText(HTML)

References(23)

References

[1]	GLOWINSKI R, MARROCCO A. Sur l’approximation par éléments finis d’ordre un et la résolution par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires[J]. Journal of Equine Veterinary Science,1975,2: 41-76.
[2]	YANG J F, YUAN X M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization[J]. Mathematics of Computation,2013,82(281): 301-329.
[3]	HE B S, YUAN X M. On non-ergodic convergence rate of douglas-rachford alternating direction method of multipliers[J]. Numerische Mathematik,2015,130(3): 567-577.
[4]	HE B S, YUAN X M. On the o(1/n) convergence rate of the Douglas-Rachford alternating direction method[J]. Society for Industrial and Applied Mathematics,2012,50(2): 700-709.
[5]	HONG M Y, LUO Z Q. On the linear convergence of the alternating direction method of multipliers[J]. Mathematical Programming,2017,162(2): 165-199.
[6]	GUO K, HAN D R, WU T T. Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints[J]. International Journal of Computer Mathematic,2017,94(8): 1653-1669.
[7]	GUO K, HAN D R, WANG Z W, et al. Convergence of ADMM for multi-block nonconvex separable optimization models[J]. Frontiers of Mathematics in China,2017,12(5): 1139-1162.
[8]	LI G Y, PONG T K. Global convergence of splitting methods for nonconvex composite optimization[J]. SIAM Journal on Optimization,2015,25(4): 2434-2460.
[9]	YU W, YIN W T, ZENG J S. Global convergence of ADMM in nonconvex nonsmooth optimization[J/OL]. Journal of Scientific Computing,2015. [2017-12-27]. https: // doi.org/ 10.1007/s10915-018-0757-z.
[10]	GABAY D. Applications of the method of multipliers tovariational inequalities[C]// In Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. Amsterdam, Holland, 1983.
[11]	LIONS P L, MERCIER B. Splitting algorithms for the sum of two nonlinear operators[J]. Siam Journal on Numerical Analysis,1979,16(6): 964-979.
[12]	ECKSTEIN J, BERTSEKAS D. On the Douglas-Rachford splitting method and proximal point algorithm for maximal monotone operators[J]. Mathematical Programming,1992,55(1): 293-318.
[13]	MARTIENT B. Regularisation d’inéquations variations par approximations succesives[J]. Rev Franaise Informat Recherche Opérationnelle,1970,4(4): 154-159.
[14]	YIN P A, XIN J. Iterative l₁-minimization for non-convex compressed sensing[J]. Journal of Computational Mathematics,2017,35(4): 439-451.
[15]	LOU Y F, YIN P H, XIN J. Point source super-resolution via non-convex l₁-based methods[J]. Journal of Scientific Computing,2016,68(3): 1082-1100.
[16]	YIN P H, LOU Y F, HE Q, et al. Minimization of 1-2 for compressed sensing[J]. SIAM Journal on Scientific Computing,2015,37(1): A536-A563.
[17]	SUN T, YIN P H, CHENG L Z, et al. Alternating direction method of multipliers with difference of convex functions[J]. Advances in Computational Mathematics,2017,44(3): 1-22.
[18]	ROCKAFELLAR R T, WETS R J. Variational Analysis [M]. Berlin: Springer-verlag, 1998.
[19]	ROCKAFELLAR R T. Convex Analysis [M]. Princeton: Princeton University Press, 1970.
[20]	ATTOUCH H, BOLTE J, REDONT P, et al. Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-ojasiewicz inequality[J]. Mathematics of Operations Research,2010,35(2): 438-457.
[21]	BOLTE J, SABACH S, TEBOULLE M. Proximal alternatinglinearized minimization for nonconvex and nonsmooth problems[J]. Mathematical Programming,2014,146(1): 459-494.
[22]	NESTEROV Y. Introductory Lectures on Convex Optimization: A Basic Course [M]. Boston: Kluwer Academic Publishers, 2004.
[23]	ATTOUCH H, BOLTE J. On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[J]. Mathematical Programming,2009,116(1): 5-16.