Abstract

We investigate how the subgradients of the value function of a discrete-time convex Bolza problem evolve over time. In particular, we develop a discrete-time version of the method of characteristics introduced by Rockafellar and Wolenski in the 2000s, by showing that the time-evolution of the subgradients of the value functions can be associated with trajectories of a discrete-time Hamiltonian system. To do so, we first prove that the value function has a dual counterpart, which corresponds to the conjugate of the value function of a suitable dual problem. We finally discuss about the qualification conditions required for our results, showing in particular that classical problems, such as the Linear Quadratic Regulator, satisfy these hypotheses.

Author information

Contact details are reproduced from the original publication and may be historical.

Mattia Solla

Dept. of Industrial and Systems Engineering, University of Southern California, Los Angeles, U.S.A.

sollasae@usc.edu

Keywords

  • Optimal control
  • Discrete-time systems
  • Convex Bolza problems
  • Linear Quadratic Regulator
  • Mixed constraints

Mathematics Subject Classification

93C55, 46N10, 49N10, 49N15

References

  1. A. Auslender, M. Teboulle: Asymptotic Cones and Functions in Optimization and Variational Inequalities, Springer, Berlin (2003).Google Scholar
  2. J. Becerril, C. Hermosilla: Optimality conditions for linear-convex optimal control problems with mixed constraints, J. Optimization Theory Appl. 194/3 (2022) 795–820.https://doi.org/10.1007/s10957-022-02049-4Google Scholar
  3. D. P. Bertsekas: Dynamic programming and optimal control, J. Oper. Res. Soc. 47/6 (1996) 833–833.https://doi.org/10.2307/3010291Google Scholar
  4. M. De Lara, L. Doyen: Sustainable Management of Natural Resources: Mathematical Models and Methods, Springer, Cham (2008).https://doi.org/10.1007/978-3-540-79074-7Google Scholar
  5. R. Goebel: Regularity of the optimal feedback and the value function in convex problems of optimal control, Set-Valued Analysis 12/1-2 (2004) 127–145.https://doi.org/10.1023/B:SVAN.0000023404.46708.60Google Scholar
  6. R. Goebel: Stationary Hamilton-Jacobi equations for convex control problems: uniqueness and duality of solutions, in: Optimal Control, Stabilization and Nonsmooth Analysis, M. de Queiroz et al. (eds.), Louisiana Conf. Mathematical Control Theory, Baton Rouge 2003, Lecture Notes in Control and Information Sciences 301, Springer, Berlin (2004) 313–322.https://doi.org/10.1007/978-3-540-39983-4_20Google Scholar
  7. R. Goebel: Convex optimal control problems with smooth Hamiltonians, SIAM J. Control Optimization 43(5) (2005) 1787–1811.https://doi.org/10.1137/S0363012902411581Google Scholar
  8. R. Goebel: Duality and uniqueness of convex solutions to stationary Hamilton-Jacobi equations, Trans. Amer. Math. Soc. 357/6 (2005) 2187–2203.https://doi.org/10.1090/S0002-9947-05-03817-1Google Scholar
  9. R. Goebel, R. T. Rockafellar: Linear-convex control and duality, in: Geometric Control And Nonsmooth Analysis, F. Ancona et al. (eds.), Proc. Conf. Rome 2006, Series on Advances in Mathematics for Applied Sciences 76, World Scientific, Hackensack (2008) 280–299.https://doi.org/10.1142/9789812776075_0017Google Scholar
  10. R. Goebel, M. Subbotin: Continuous time linear quadratic regulator with control constraints via convex duality, IEEE Trans. Automatic Control 52/5 (2007) 886–892.https://doi.org/10.1109/TAC.2007.895915Google Scholar
  11. C. Hermosilla, P. R. Wolenski: Constrained and impulsive linear quadratic control problems, in: Proc. 20th World Congress IFAC, Vol. 50 (2017) 1637–1642.https://doi.org/10.1016/j.ifacol.2017.08.330Google Scholar
  12. C. Hermosilla, P. R. Wolenski: A characteristic method for fully convex Bolza problems over arcs of bounded variation, SIAM J. Control Optimization 57/4 (2019) 2873–2901.https://doi.org/10.1137/18M1191592Google Scholar
  13. R. T. Rockafellar: Conjugate convex functions in optimal control and the calculus of variations, J. Math. Analysis Appl. 32/1 (1970) 174–222.https://doi.org/10.1016/0022-247X(70)90324-0Google Scholar
  14. R. T. Rockafellar: Convex Analysis, Princeton University Press, Princeton (1970).https://doi.org/10.1515/9781400873173Google Scholar
  15. R. T. Rockafellar: Existence and duality theorems for convex problems of Bolza, Trans. Amer. Math. Soc. 159 (1971) 1–40.https://doi.org/10.1090/S0002-9947-1971-0282283-0Google Scholar
  16. R. T. Rockafellar: State constraints in convex control problems of Bolza, SIAM J. Control Optimization 10/4 (1972) 691–715.https://doi.org/10.1137/0310051Google Scholar
  17. R. T. Rockafellar: Saddle points of Hamiltonian systems in convex problems of Lagrange, J. Optimization Theory Appl. 12 (1973) 367–390.https://doi.org/10.1007/BF00940418Google Scholar
  18. R. T. Rockafellar: Dual problems of Lagrange for arcs of bounded variation, in: Calculus of Variations and Control Theory, Proc. Sympos., Math. Res. Center, Univ. Wisconsin, Madison 1975, Academic Press, New York (1976) 155–192.Google Scholar
  19. R. T. Rockafellar: Linear-quadratic programming and optimal control, SIAM J. Control Optimization 25/3 (1987) 781–814.https://doi.org/10.1137/0325045Google Scholar
  20. R. T. Rockafellar: Hamilton-Jacobi theory and parametric analysis in fully convex problems of optimal control, J. Global Optimization 28/3-4 (2004) 419–431.https://doi.org/10.1023/B:JOGO.0000026459.51919.0eGoogle Scholar
  21. R. T. Rockafellar, R. J-B Wets: Deterministic and stochastic optimization problems of Bolza type in discrete time, Stochastics 10/3-4 (1983) 273–312.https://doi.org/10.1080/17442508308833276Google Scholar
  22. R. T. Rockafellar, P. R. Wolenski: Convexity in Hamilton-Jacobi theory. I: Dynamics and duality, SIAM J. Control Optimization 32/2 (2000) 442–470.Google Scholar
  23. R. T. Rockafellar, P. R. Wolenski: Convexity in Hamilton-Jacobi theory. I: Envelope representations, SIAM J. Control Optimization 39/5 (2000) 1351–1372.https://doi.org/10.1137/S0363012998345378Google Scholar
  24. J. O. Royset, R. J-B Wets: An Optimization Primer, Springer, Berlin (2022).https://doi.org/10.1007/978-3-030-76275-9Google Scholar
  25. A. Seeger: Subgradients of optimal-value functions in dynamic programming: the case of convex systems without optimal paths, Math. Oper. Res. 21/3 (1996) 555–575.https://doi.org/10.1287/moor.21.3.555Google Scholar

Suggested citation

J. Deride, C. Hermosilla, M. Solla. “Subgradient Evolution of Value Functions of Discrete-Time Convex Bolza Problems.” Journal of Convex Analysis 33 (2026), No. 1&2, 207–225. https://doi.org/10.68381/jca33014

Download BibTeXPublished by Heldermann Verlag.

Published by Heldermann Verlag, 2026. Rights now held by Banach Press LLC.