Abstract

In a Hilbert space H\cH, we study the stabilization in finite-time of the trajectories generated by a continuous (in time tt) damped inertial dynamic system. The potential function f ⁣:H→Rf\colon \cH \to \mathbb{R} to be minimized is supposed to be differentiable, not necessarily convex. It enters the dynamic via its gradient. The damping results from the joint action of dry friction, viscous friction, and a geometric damping driven by the Hessian of ff. The dry friction damping function ϕ ⁣:H→R+\phi\colon \cH \to \mathbb{R}_+, which is convex and continuous with a sharp minimum at the origin (typically ϕ(x)=r∥x∥\phi(x) = r \|x\| with r>0r>0), enters the dynamic via its subdifferential. It acts as a soft threshold operator on the velocities, and is at the origin of the stabilization property in finite-time. The Hessian driven damping, which enters the dynamics in the form ∇2f(x(t))x˙(t)\nabla^2 f(x(t))\dot{x}(t), permits to control and attenuate the oscillations which occur naturally with the inertial effect. We give two different proofs, in a finite dimensional setting, of the existence of strong solutions of this second-order differential inclusion. One is based on a fixed point argument and Leray-Schauder theorem, the other one uses the Yosida approximation technique together with the Mosco convergence. We also give an existence and uniqueness result in a general Hilbert framework by assuming that the Hessian of the function ff is Lipschitz continuous on the bounded sets of H\cH. Then, we study the convergence properties of the trajectories as t→+∞t \to +\infty, and show their stabilization property in finite-time. The convergence results tolerate the presence of perturbations (or errors) under the sole assumption of their asymptotic convergence to zero. The study is extended to the case of a nonsmooth convex function ff by using Moreau's envelope.

Author information

Contact details are reproduced from the original publication and may be historical.

Keywords

  • Damped inertial dynamics
  • differential inclusion
  • dry friction
  • Hessian-driven damping
  • finite-time stabilization

Mathematics Subject Classification

37N40, 34A60, 34G25, 49K24, 70F40

References

  1. S. Adly: A Variational Approach to Nonsmooth Dynamics. Applications in Unilateral Mechanics and Electronics, Springer Briefs in Mathematics, Springer, Berlin (2017).https://doi.org/10.1007/978-3-319-68658-5Google Scholar
  2. S. Adly, H. Attouch: Finite convergence of proximal-gradient inertial algorithms with dry friction damping, hal.archives-ouvertes.fr/hal-02388038 (2019).Google Scholar
  3. S. Adly, H. Attouch: Finite convergence of proximal-gradient inertial algorithms combining dry friction with Hessian-driven damping, SIAM J. Optimization 30/3 (2020) 2134–2162.https://doi.org/10.1137/19M1307779Google Scholar
  4. S. Adly, H. Attouch, A. Cabot: Finite time stabilization of nonlinear oscillators subject to dry friction, in: Nonsmooth Mechanics and Analysis, P. Alart, O. Maisonneuve, R. T. Rockafellar (eds.), Advances in Mechanics and Mathematics 12, Springer, Boston (2006) 289–304.https://doi.org/10.1007/0-387-29195-4_24Google Scholar
  5. F. Álvarez: On the minimizing property of a second-order dissipative system in Hilbert spaces, SIAM J. Control Optimization 38/4 (2000) 1102–1119.https://doi.org/10.1137/S0363012998335802Google Scholar
  6. F. Álvarez, H. Attouch, J. Bolte, P. Redont: A second-order gradient-like dissipative dynamical system with Hessian-driven damping, J. Math. Pures Appl. 81/8 (2002) 747–779.https://doi.org/10.1016/S0021-7824(01)01253-3Google Scholar
  7. V. Apidopoulos, J.-F. Aujol, Ch. Dossal: Convergence rate of inertial forward-backward algorithm beyond Nesterov’s rule, Math. Programming 180 (2020) 137–156.https://doi.org/10.1007/s10107-018-1350-9Google Scholar
  8. H. Attouch: Variational Convergence for Functions and Operators, Pitman Advanced Publishing Program, Applicable Mathematics Series, Pitman, London (1984).Google Scholar
  9. H. Attouch, G. Buttazzo, G. Michaille: Variational Analysis in Sobolev and BV Spaces. Applications to PDEs and Optimization, 2nd edition, MOS/SIAM Series on Optimization 17, SIAM, Philadelphia (2014).https://doi.org/10.1137/1.9781611973488Google Scholar
  10. H. Attouch, A. Cabot: Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity, J. Diff. Equations 263 (2017) 5412–5458.https://doi.org/10.1016/j.jde.2017.06.024Google Scholar
  11. H. Attouch, A. Cabot: Convergence rates of inertial forward-backward algorithms, SIAM J. Optimization 28/1 (2018) 849–874.https://doi.org/10.1137/17M1114739Google Scholar
  12. H. Attouch, A. Cabot, Z. Chbani, H. Riahi: Rate of convergence of inertial gradient dynamics with time-dependent viscous damping coefficient, Evolution Equations and Control Theory 7/3 (2018) 353–371.https://doi.org/10.3934/eect.2018018Google Scholar
  13. H. Attouch, Z. Chbani, J. Fadili, H. Riahi: First-order optimization algorithms via inertial systems with Hessian driven damping, Math. Programming (2020).https://doi.org/10.1007/s10107-020-01591-1Google Scholar
  14. H. Attouch, Z. Chbani, J. Peypouquet, P. Redont: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity, Math. Programming, Ser. B 168 (2018) 123–175.https://doi.org/10.1007/s10107-016-0992-8Google Scholar
  15. H. Attouch, Z. Chbani, H. Riahi: Fast proximal methods via time scaling of damped inertial dynamics, SIAM J. Optimization 29/3 (2019) 2227–2256.https://doi.org/10.1137/18M1230207Google Scholar
  16. H. Attouch, Z. Chbani, H. Riahi: Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3, ESAIM-COCV 25 (2019), art. no. 2.https://doi.org/10.1051/cocv/2017083Google Scholar
  17. H. Attouch, X. Goudou, P. Redont: The heavy ball with friction method. The continuous dynamical system, global exploration of the local minima of a real-valued function by asymptotical analysis of a dissipative dynamical system, Commun. Contemp. Math. 2/1 (2000) 1–34.https://doi.org/10.1142/S0219199700000025Google Scholar
  18. H. Attouch, P. E. Maingé, P. Redont: A second-order differential system with Hessian-driven damping; Application to non-elastic shock laws, Diff. Equations Appl. 4/1 (2012) 27–65.https://doi.org/10.7153/dea-04-04Google Scholar
  19. H. Attouch, J. Peypouquet: The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than 1/k 2, SIAM J. Optimization 26/3 (2016) 1824–1834.https://doi.org/10.1137/15M1046095Google Scholar
  20. H. Attouch, J. Peypouquet, P. Redont: Fast convex minimization via inertial dynamics with Hessian driven damping, J. Diff. Equations 261 (2016) 5734–5783.https://doi.org/10.1016/j.jde.2016.08.020Google Scholar
  21. H. Attouch, B. F. Svaiter: A continuous dynamical Newton-Like approach to solving monotone inclusions, SIAM J. Control Optimization 49/2 (2011) 574–598.https://doi.org/10.1137/100784114Google Scholar
  22. J.-F. Aujol, Ch. Dossal: Stability of over-relaxations for the Forward-Backward algorithm, application to FISTA, SIAM J. Optimization 25/4 (2015) 2408–2433.https://doi.org/10.1137/140994964Google Scholar
  23. J.-F. Aujol, Ch. Dossal: Optimal rate of convergence of an ODE associated to the Fast Gradient Descent schemes for b > 0, hal.inria.fr/hal-01547251v2 (2017).Google Scholar
  24. B. Baji, A. Cabot: An inertial proximal algorithm with dry friction: finite convergence results, Set-Valued Analysis 9(1 (2006) 1–23.https://doi.org/10.1007/s11228-005-0002-4Google Scholar
  25. H. Bauschke, P. L. Combettes: Convex Analysis and Monotone Operator Theory in Hilbert spaces, CMS Books in Mathematics, Springer, Berlin (2011).https://doi.org/10.1007/978-1-4419-9467-7Google Scholar
  26. R. I. Bot, E. R. Csetnek: Second order forward-backward dynamical systems for monotone inclusion problems, SIAM J. Control Optimization 54/3 (2016) 1423–1443.https://doi.org/10.1137/15M1012657Google Scholar
  27. R. I. Bot, E. R. Csetnek, S. C. László: A second order dynamical approach with variable damping to nonconvex smooth minimization, Applicable Analysis 99/3 (2018) 361–378.https://doi.org/10.1080/00036811.2018.1495330Google Scholar
  28. H. Brézis: Opérateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert, Mathematics Studies 5, North Holland, Amsterdam (1972).Google Scholar
  29. C. Castera, J. Bolte, C. Févotte, E. Pauwels: An inertial Newton algorithm for deep learning, hal.archives-ouvertes.fr/hal-02140748 (2019).Google Scholar
  30. A. Chambolle, Ch. Dossal: On the convergence of the iterates of the fast iterative shrinkage thresholding algorithm, J. Optim. Theory Appl. 166 (2015) 968–982.https://doi.org/10.1007/s10957-015-0746-4Google Scholar
  31. A. Chambolle, T. Pock: An introduction to continuous optimization for imaging, Acta Numerica 25 (2016) 161–319.https://doi.org/10.1017/S096249291600009XGoogle Scholar
  32. R. May: Asymptotic for a second-order evolution equation with convex potential and vanishing damping term, Turkish J. Math. 41/3 (2017) 681–685.https://doi.org/10.3906/mat-1512-28Google Scholar
  33. U. Mosco: Convergence of convex sets and of solutions of variational inequalities, Advances Math. 3 (1969) 510–585.https://doi.org/10.1016/0001-8708(69)90009-7Google Scholar
  34. U. Mosco: On the continuity of the Young-Fenchel transform, J. Math. Anal. Appl. 35/3 (1971) 518–535.https://doi.org/10.1016/0022-247X(71)90200-9Google Scholar
  35. B. T. Polyak: Some methods of speeding up the convergence of iterative methods, Z. Vylist Math. Fiz. 4 (1964) 1–17.https://doi.org/10.1016/0041-5553(64)90137-5Google Scholar
  36. B. T. Polyak: Introduction to Optimization, Optimization Software, New York (1987).Google Scholar
  37. H. Schaefer: Über die Method der a priori-Schranken, Math. Annalen 126 (1955) 415–416.https://doi.org/10.1007/BF01362380Google Scholar
  38. B. Shi, S. S. Du, M. I. Jordan, W. J. Su: Understanding the acceleration phenomenon via high-resolution differential equations, arXiv: 1810.08907 (2018).Google Scholar
  39. W. Su, S. Boyd, E. J. Candès: A differential equation for modeling Nesterov’s accelerated gradient method, J. Machine Learning Res. 17 (2016) 1–43.Google Scholar

Suggested citation

S. Adly, H. Attouch. “Finite-Time Stabilization of Continuous Inertial Dynamics Combining Dry Friction with Hessian-Driven Damping.” Journal of Convex Analysis 28 (2021), No. 2, 281–310. https://doi.org/10.68381/jca28019

Download BibTeXPublished by Heldermann Verlag.

Published by Heldermann Verlag, 2021. Rights now held by Banach Press LLC.