Abstract

We study several aspects of the dynamic programming approach to optimal control of abstract evolution equations, including a class of semilinear partial differential equations. We introduce and prove a verification theorem which provides a sufficient condition for optimality. Moreover we prove sub- and superoptimality principles of dynamic programming and present an explicit construction of ε-optimal controls.

Author information

Contact details are reproduced from the original publication and may be historical.

Giorgio Fabbri

Dipartimento di Studi Economici, Università di Napoli Parthenope, Napoli, Italy
and: School of Mathematics and Statistics, UNSW, Sydney, Australia

giorgio.fabbri@uniparthenope.it

Fausto Gozzi

Dip. di Scienze Economiche ed Aziendali, Libera Università Internazionale degli Studi Sociali, Viale Pola 12, 00198 Roma, Italy
and: Centro De Giorgi, Scuola Normale Superiore, Pisa, Italy

fgozzi@luiss.it

Andrzej Święch

School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, U.S.A.

swiech@math.gatech.edu

Keywords

  • Optimal control of PDE
  • verification theorem
  • dynamic programming
  • epsilon-optimal controls
  • Hamilton-Jacobi-Bellman equations

Mathematics Subject Classification

35R15, 49L20, 49L25, 49K20

References

  1. S. Aniţa: Analysis and Control of Age-Dependent Population Dynamics, Kluwer, Dordrecht (2001).https://doi.org/10.1007/978-94-015-9436-3Google Scholar
  2. V. Barbu: Optimal feedback controls for a class of nonlinear distributed parameter systems, SIAM J. Control Optimization 21(6) (1983) 871–894.https://doi.org/10.1137/0321053Google Scholar
  3. V. Barbu: Hamilton-Jacobi equations and nonlinear control problems, J. Math. Anal. Appl. 120(2) (1986) 494–509.https://doi.org/10.1016/0022-247X(86)90171-XGoogle Scholar
  4. V. Barbu: Analysis and Control of Nonlinear Infinite-Dimensional Systems, Mathematics in Science and Engineering 190, Academic Press, Boston (1993).Google Scholar
  5. V. Barbu: Approximation Hamilton-Jacobi equations and suboptimal feedback controllers, in: Nonlinear Analysis and Applications (Warsaw, 1994), N. Kenmochi et al. (ed.), GAKUTO Int. Ser., Math. Sci. Appl. 7, Gakkōtosho, Tokyo (1996) 27–47.Google Scholar
  6. V. Barbu, E. N. Barron, R. Jensen: The necessary conditions for optimal control in Hilbert spaces, J. Math. Anal. Appl. 133(1) (1988) 151–162.https://doi.org/10.1016/0022-247X(88)90372-1Google Scholar
  7. V. Barbu, G. Da Prato: Hamilton-Jacobi Equations in Hilbert Spaces, Research Notes in Mathematics 86, Pitman, Boston (1983).Google Scholar
  8. V. Barbu, G. Da Prato: Hamilton-Jacobi equations and synthesis of nonlinear control processes in Hilbert spaces, J. Differ. Equations 48(3) (1983) 350–372.https://doi.org/10.1016/0022-0396(83)90099-2Google Scholar
  9. V. Barbu, G. Da Prato: Hamilton-Jacobi equations in Hilbert spaces: variational and semigroup approach, Ann. Mat. Pura Appl. 142(4) (1985) 303–349.https://doi.org/10.1007/BF01766599Google Scholar
  10. V. Barbu, G. Da Prato: A note on a Hamilton-Jacobi equation in Hilbert space, Nonlinear Anal. 9(12) (1985) 1337–1345.https://doi.org/10.1016/0362-546X(85)90094-XGoogle Scholar
  11. V. Barbu, G. Da Prato, C. Popa: Existence and uniqueness of the dynamic programming equation in Hilbert space, Nonlinear Anal. 7(3) (1983) 283–299.https://doi.org/10.1016/0362-546X(83)90073-1Google Scholar
  12. M. Bardi, I. Capuzzo-Dolcetta: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Systems & Control: Foundations & Applications, Birkhäuser, Boston (1997).https://doi.org/10.1007/978-0-8176-4755-1Google Scholar
  13. E. Barucci, F. Gozzi: Investment in a vintage capital model, Res. Econ. 52 (1998) 159–188.https://doi.org/10.1006/reec.1997.0159Google Scholar
  14. E. Barucci, F. Gozzi: Technology adoption and accumulation in a vintage capital model, J. Econ. 74(1) (2001) 1–30.https://doi.org/10.1007/BF01231214Google Scholar
  15. A. Bensoussan, G. Da Prato, M. C. Delfour, S. K. Mitter: Representation and Control of Infinite Dimensional Systems, Birkhäuser, Boston (1992).https://doi.org/10.1007/978-1-4612-2750-2Google Scholar
  16. J. M. Borwein, D. Preiss: A smooth variational principle with applications to subdifferentiability and to differentiability of convex functions, Trans. Amer. Math. Soc. 303(2) (1987) 517–527.https://doi.org/10.1090/S0002-9947-1987-0902782-7Google Scholar
  17. P. Cannarsa: Regularity properties of solutions to Hamilton-Jacobi equations in infinite dimensions and nonlinear optimal control, Differ. Integral Equ. 2(4) (1989) 479–493.https://doi.org/10.57262/die/1371648439Google Scholar
  18. P. Cannarsa, O. Carjă: On the Bellman equation for the minimum time problem in infinite dimensions, SIAM J. Control Optimization 43(2) (2004) 532–548https://doi.org/10.1137/S0363012902419011Google Scholar
  19. P. Cannarsa, G. Da Prato: Nonlinear optimal control with infinite horizon for distributed parameter systems and stationary Hamilton-Jacobi equations, SIAM J. Control Optimization 27(4) (1989) 861–875.https://doi.org/10.1137/0327046Google Scholar
  20. P. Cannarsa, G. Da Prato: Some results on non-linear optimal control problems and Hamilton-Jacobi equations in infinite dimensions, J. Funct. Anal. 90(1) (1990) 27–47.https://doi.org/10.1016/0022-1236(90)90079-ZGoogle Scholar
  21. P. Cannarsa, G. Di Blasio: A direct approach to infinite-dimensional Hamilton-Jacobi equations and applications to convex control with state constraints, Differ. Integral Equ. 8(2) (1995) 225–246.https://doi.org/10.57262/die/1369083467Google Scholar
  22. P. Cannarsa, F. Gozzi, H. M. Soner: A dynamic programming approach to nonlinear boundary control problems of parabolic type, J. Funct. Anal. 117(1) (1993) 25–61.https://doi.org/10.1006/jfan.1993.1122Google Scholar
  23. P. Cannarsa, H. Frankowska: Value function and optimality conditions for semilinear control problems, Appl. Math. Optimization 26(2) (1992) 139–169.https://doi.org/10.1007/BF01189028Google Scholar
  24. P. Cannarsa, H. Frankowska: Value function and optimality condition for semilinear control problems. II: parabolic case, Appl. Math. Optimization 33(1) (1996) 1–33.https://doi.org/10.1007/BF01187960Google Scholar
  25. P. Cannarsa, C. Sinestrari: Semiconcave Functions, Hamilton-Jacobi Equations, and Optimal Control, Progress in Nonlinear Differential Equations and their Applications 58, Birkhäuser, Boston (2004).https://doi.org/10.1007/b138356Google Scholar
  26. P. Cannarsa, M. E. Tessitore: Optimality conditions for boundary control problems of parabolic type, in: Control and Estimation of Distributed Parameter Systems: Nonlinear Phenomena (Vorau, 1993), W. Desch et al. (ed.), Int. Ser. Numer. Math. 118, Birkhäuser, Basel (1994) 79–96.https://doi.org/10.1007/978-3-0348-8530-0_5Google Scholar
  27. P. Cannarsa, M. E. Tessitore: Infinite dimensional Hamilton–Jacobi equations and Dirichlet boundary control problems of parabolic type, SIAM J. Control Optimization 34(6) (1996) 1831–1847.https://doi.org/10.1137/S0363012994263354Google Scholar
  28. F. H. Clarke, Yu. S. Ledyaev, E. D. Sontag, A. I. Subbotin: Asymptotic controllability implies feedback stabilization, IEEE Trans. Autom. Control 42(10) (1997) 1394–1407.https://doi.org/10.1109/9.633828Google Scholar
  29. F. H. Clarke, Yu. S. Ledyaev, R. J. Stern, P. R. Wolenski: Nonsmooth Analysis and Control Theory, Springer, New York (1998).Google Scholar
  30. M. G. Crandall, H. Ishii, P. L. Lions: User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc., New Ser. 27(1) (1992) 1–67.https://doi.org/10.1090/S0273-0979-1992-00266-5Google Scholar
  31. M. G. Crandall, P. L. Lions: Hamilton-Jacobi equations in infinite dimensions. IV. Hamiltonians with unbounded linear terms, J. Funct. Anal. 90 (1990) 237–283.https://doi.org/10.1016/0022-1236(90)90084-XGoogle Scholar
  32. M. G. Crandall, P. L. Lions: Hamilton-Jacobi equations in infinite dimensions. V. Unbounded linear terms and B-continuous solutions, J. Funct. Anal. 97 (1991) 417–465.https://doi.org/10.1016/0022-1236(91)90010-3Google Scholar
  33. M. G. Crandall, P. L. Lions: Hamilton-Jacobi equations in infinite dimensions. VI. Nonlinear A and Tataru’s method refined, in: Evolution Equations, Control Theory, and Biomathematics (Han sur Lesse, 1991), P. Clément et al. (ed.), Lect. Notes Pure Appl. Math. 155, Dekker, New York (1993) 51–89.Google Scholar
  34. M. G. Crandall, P. L. Lions: Hamilton-Jacobi equations in infinite dimensions. VII. The HJB equation is not always satisfied, J. Funct. Anal. 125 (1994) 111–148.https://doi.org/10.1006/jfan.1994.1119Google Scholar
  35. G. Di Blasio: Global solutions for a class of Hamilton-Jacobi equations in Hilbert spaces, Numer. Funct. Anal. Optimization 8(3–4) (1985/86) 261–300.https://doi.org/10.1080/01630568608816214Google Scholar
  36. I. Ekeland, G. Lebourg: Generic Frechet-differentiability and perturbed optimization problems in Banach spaces, Trans. Amer. Math. Soc. 224 (1977) 193–216.https://doi.org/10.1090/S0002-9947-1976-0431253-2Google Scholar
  37. G. Fabbri: A viscosity solution approach to the infinite-dimensional HJB equation related to a boundary control problem in a transport equation, SIAM J. Control Optimization 47(2) (2008) 1022–1052.https://doi.org/10.1137/050638813Google Scholar
  38. S. Faggian, F. Gozzi: On the dynamic programming approach for optimal control problems of PDE’s with age structure, Math. Popul. Stud. 11(3–4) (2004) 233–270.https://doi.org/10.1080/08898480490513625Google Scholar
  39. S. Federico, B. Goldys, F. Gozzi: HJB Equations for the optimal control of differential equations with delays and state constraints II: Optimal feedbacks and approximations, arXiv:0905.1825 (2009).https://doi.org/10.1137/09076742XGoogle Scholar
  40. G. Feichtinger, A. Prskawetz, V. M. Veliov: Age-structured optimal control in population economics, Math. Popul. Stud. 65(4) (2004) 373–387.https://doi.org/10.1016/j.tpb.2003.07.006Google Scholar
  41. W. Fleming, R. W. Rishel: Deterministic and Stochastic Optimal Control, Applications of Mathematics 1, Springer, Berlin (1975).https://doi.org/10.1007/978-1-4612-6380-7_1Google Scholar
  42. W. Fleming, H. M. Soner: Controlled Markov Processes and Viscosity Solutions, 2nd Ed., Stochastic Modelling and Applied Probability 25, Springer, New York (2006).Google Scholar
  43. F. Gozzi, S. S. Sritharan, A. Święch: Viscosity solutions of dynamic-programming equations for the optimal control of the two-dimensional Navier-Stokes equations, Arch. Ration. Mech. Anal. 163(4) (2002) 295–327.https://doi.org/10.1007/s002050200203Google Scholar
  44. M. Iannelli: Mathematical Theory of Age-Structured Population Dynamics, Giardini, Pisa (1995).Google Scholar
  45. M. Iannelli, M. Martcheva, F. A. Milner: Gender-Structured Population Modeling, SIAM, Philadelphia (2005).https://doi.org/10.1137/1.9780898717488Google Scholar
  46. H. Ishii: Viscosity solutions for a class of Hamilton–Jacobi equations in Hilbert spaces, J. Funct. Anal. 105 (1992) 301–341.https://doi.org/10.1016/0022-1236(92)90081-SGoogle Scholar
  47. H. Ishii, S. Koike: On ε-optimal controls for state constraint problems, Ann. Inst. H. Poincaré, Anal. Non Linéaire 17(4) (2000) 473–502.https://doi.org/10.1016/s0294-1449(00)00114-1Google Scholar
  48. D. Kelome: Viscosity Solutions of Second Order Equations in a Separable Hilbert Space and Applications to Stochastic Optimal Control, Ph.D. Thesis, Georgia Institute of Technology, Atlanta (2002).Google Scholar
  49. D. Kelome, A. Święch: Perron’s method and the method of relaxed limits for “unbounded” PDE in Hilbert spaces, Stud. Math. 176(3) (2006) 249–277.https://doi.org/10.4064/sm176-3-4Google Scholar
  50. M. Kocan, P. Soravia: A viscosity approach to infinite-dimensional Hamilton-Jacobi equations arising in optimal control with state constraints, SIAM J. Control Optimization 36 (1998) 1348–1375.https://doi.org/10.1137/S0363012996301622Google Scholar
  51. X. Y. Li, J. M. Yong: Optimal Control Theory for Infinite-Dimensional Systems, Birkhäuser, Basel (1994).Google Scholar
  52. P. D. Loewen: Optimal control via Nonsmooth Analysis, CRM Proceedings & Lecture Notes 2, AMS, Providence (1993).https://doi.org/10.1090/crmp/002Google Scholar
  53. M. Renardy: Polar decomposition of positive operators and problem of Crandall and Lions, Appl. Anal. 57(3–4) (1995) 383–385.https://doi.org/10.1080/00036819508840358Google Scholar
  54. S. Saks: Theory of the Integral, 2nd rev. Ed., Dover, New York (1964).Google Scholar
  55. K. Shimano: A class of Hamilton-Jacobi equations with unbounded coefficients in Hilbert spaces, Appl. Math. Optimization 45(1) (2002) 75–98.https://doi.org/10.1007/s00245-001-0028-4Google Scholar
  56. S. S. Sritharan: Dynamic programming of the Navier-Stokes equations, Syst. Control Lett. 16(4) (1991) 299–307.https://doi.org/10.1016/0167-6911(91)90020-FGoogle Scholar
  57. A. Święch: Sub- and superoptimality principles of dynamic programming revisited, Nonlinear Anal. 26(8) (1996) 1429–1436.https://doi.org/10.1016/0362-546X(94)00348-LGoogle Scholar
  58. D. Tataru: Viscosity solutions of Hamilton–Jacobi equations with unbounded linear terms, J. Math. Anal. Appl. 163 (1992) 345–392.https://doi.org/10.1016/0022-247X(92)90256-DGoogle Scholar
  59. D. Tataru: Viscosity solutions for the dynamic programming equation, Appl. Math. Optimization 25 (1992) 109–126.https://doi.org/10.1007/BF01182476Google Scholar
  60. R. Vinter: Optimal Control, Systems & Control: Foundations & Applications, Birkhäuser, Boston (2000).Google Scholar
  61. J. Yong, X. Y. Zhou: Stochastic Controls. Hamiltonian Systems and HJB Equations, Springer, New York (1999).https://doi.org/10.1007/978-1-4612-1466-3Google Scholar
  62. X. Y. Zhou: Verification theorems within the framework of viscosity solutions, J. Math. Anal. Appl. 177(1) (1993) 208–225.https://doi.org/10.1006/jmaa.1993.1253Google Scholar

Suggested citation

G. Fabbri, F. Gozzi, A. Święch. “Verification Theorem and Construction of ε-Optimal Controls for Control of Abstract Evolution Equations.” Journal of Convex Analysis 17 (2010), No. 2, 611–642. https://doi.org/10.68381/jca17041

Download BibTeXPublished by Heldermann Verlag.

Published by Heldermann Verlag, 2010. Rights now held by Banach Press LLC.