Abstract

This paper studies the dynamic programming principle for general convex stochastic optimization problems introduced by R. T. Rockafellar and R. J-B Wets [Nonanticipativity and L1L^1-martingales in stochastic optimization problems, Math. Programming Studies 6 (1976) 170–187]. We extend the applicability of the theory by relaxing compactness and boundedness assumptions. In the context of financial mathematics, the relaxed assumptions are satisfied under the well-known no-arbitrage condition and the "reasonable asymptotic elasticity" condition of the utility function. Besides financial mathematics, we obtain several new results in linear and nonlinear stochastic programming and stochastic optimal control.

Author information

Contact details are reproduced from the original publication and may be historical.

Ari-Pekka Perkkiö

Mathematics Institute, Ludwig-Maximilian University, Munich, Germany

a.perkkioe@lmu.de

Keywords

  • Dynamic programming
  • stochastic programming
  • convexity

Mathematics Subject Classification

46N10, 90C39, 93E20

References

  1. E. M. L. Beale: On minimizing a convex function subject to linear inequalities, J. Royal Stat. Soc. Ser. B 17 (1955) 173–184; discussion, ibid. (1955) 194–203.https://doi.org/10.1111/j.2517-6161.1955.tb00191.xGoogle Scholar
  2. R. Bellman: Dynamic Programming, Reprint of the 1957 ed., Princeton Landmarks in Mathematics, Princeton University Press, Princeton (2010).Google Scholar
  3. D. P. Bertsekas: Dynamic Programming and Stochastic Control, Mathematics in Science and Engineering Vol. 125, Academic Press, New York (1976).Google Scholar
  4. D. P. Bertsekas: Reinforcement Learning and Optimal Control, Athena Scientific Optimization and Computation Series, Athena Scientific, Nashua (2019).Google Scholar
  5. D. P. Bertsekas, S. E. Shreve: Stochastic Optimal Control, Mathematics in Science and Engineering Vol. 139, Academic Press, New York (1978).Google Scholar
  6. S. Biagini, T. Pennanen, A.-P. Perkkiö: Duality and optimality conditions in stochastic optimization and mathematical finance, J. Convex Analysis 25/2 (2018) 403–420.https://doi.org/10.68381/jca25028Google Scholar
  7. J.-M. Bismut: Intégrales convexes et probabilités, J. Math. Analysis Appl. 42 (1973) 639–673.https://doi.org/10.1016/0022-247X(73)90170-4Google Scholar
  8. P. Carpentier, J.-P. Chancelier, G. Cohen, M. De Lara: Stochastic Multi-Stage Optimization, Probability Theory and Stochastic Modelling Vol. 75, Springer, Cham (2015).https://doi.org/10.1007/978-3-319-18138-7Google Scholar
  9. C. Choirat, C. Hess, R. Seri: A functional version of the Birkhoff ergodic theorem for a normal integrand: A variational approach, Ann Probability 31/1 (2003) 63–92.https://doi.org/10.1214/aop/1046294304Google Scholar
  10. G. B. Dantzig: Linear programming under uncertainty, Management Sci. 1 (1955) 197–206.https://doi.org/10.1287/mnsc.1.3-4.197Google Scholar
  11. E. B. Dynkin: Probabilistic concave dynamic programming, Mat. Sb. (N.S.) 87/129 (1972) 490–503.Google Scholar
  12. E. B. Dynkin, I. V. Evstigneev: Regular conditional expectation of correspondences, Teor. Ver. Primen. 21/2 (1976) 334–347.https://doi.org/10.1137/1121037Google Scholar
  13. I. V. Evstigneev: Measurable selection and dynamic programming, Math. Oper. Res. 1/3 (1976) 267–272.https://doi.org/10.1287/moor.1.3.267Google Scholar
  14. W. H. Fleming, R. W. Rishel: Deterministic and Stochastic Optimal Control, Applications of Mathematics Vol. 1, Springer, Berlin (1975).https://doi.org/10.1007/978-1-4612-6380-7_1Google Scholar
  15. F. Hiai, H. Umegaki: Integrals, conditional expectations, and martingales of multivalued functions, J. Multivar. Analysis 7/1 (1977) 149–182.https://doi.org/10.1016/0047-259X(77)90037-9Google Scholar
  16. J. Jacod, A. N. Shiryaev: Local martingales and the fundamental asset pricing theorems in the discrete-time case, Finance Stochastics 2/3 (1998) 259–273.https://doi.org/10.1007/s007800050040Google Scholar
  17. O. Kallenberg: Foundations of Modern Probability, 2nd ed., Springer, New York (2002).https://doi.org/10.1007/978-1-4757-4015-8Google Scholar
  18. D. Kramkov, W. Schachermayer: The condition on the asymptotic elasticity of utility functions and optimal investment in incomplete markets, Ann Appl. Probab. 9/3 (1999) 904–950.https://doi.org/10.1214/aoap/1029962818Google Scholar
  19. R. C. Merton: Lifetime portfolio selection under uncertainty: The continuous-time case, Rev. Econ. Stat. 3 (1969) 373–413.https://doi.org/10.2307/1926560Google Scholar
  20. P. Olsen: When is a multistage stochastic programming problem well-defined?, SIAM J. Control Optim. 14/3 (1976) 518–527.https://doi.org/10.1137/0314034Google Scholar
  21. T. Pennanen, A.-P. Perkkiö: Stochastic programs without duality gaps, Math. Programming 136/1 (2012) 91–110.https://doi.org/10.1007/s10107-012-0552-9Google Scholar
  22. T. Pennanen, A.-P. Perkkiö: Convex duality in optimal investment and contingent claim valuation in illiquid markets, Finance Stochastics 22/4 (2018) 733–771.https://doi.org/10.1007/s00780-018-0372-8Google Scholar
  23. T. Pennanen, A.-P. Perkkiö: Duality in convex stochastic optimization, arxiv: 2205.02045 (2022).Google Scholar
  24. T. Pennanen, A.-P. Perkkiö: Optimal stopping without Snell envelopes, Proc. Amer. Math. Soc. (2023).Google Scholar
  25. T. Pennanen, A.-P. Perkkiö, M. Rásonyi: Existence of solutions in non-convex dynamic programming and optimal investment, Math. Financial Econ. 11/2 (2017) 173–188.https://doi.org/10.1007/s11579-016-0176-6Google Scholar
  26. A.-P. Perkkiö: Stochastic programs without duality gaps for objectives without a lower bound, manuscript (2016).Google Scholar
  27. G. Peskir, A. Shiryaev: Optimal Stopping and Free-Boundary Problems, Lectures in Mathematics at the ETH Zürich, Birkhäuser, Basel (2006).https://doi.org/10.1007/978-3-7643-7390-0Google Scholar
  28. M. Rásonyi, L. Stettner: On utility maximization in discrete-time financial market models, Ann. Appl. Probab. 15/2 (2005) 1367–1395.https://doi.org/10.1214/105051605000000089Google Scholar
  29. R. T. Rockafellar: Convex Analysis, Princeton Mathematical Series Vol. 28, Princeton University Press, Princeton (1970).Google Scholar
  30. R. T. Rockafellar, R. J.-B. Wets: Nonanticipativity and L1 -martingales in stochastic optimization problems, Math. Programming Studies 6 (1976) 170–187.https://doi.org/10.1007/BFb0120750Google Scholar
  31. R. T. Rockafellar, R. J.-B. Wets: The optimal recourse problem in discrete time: L1 - multipliers for inequality constraints, SIAM J. Control Optimization 16/1 (1978) 16–36.https://doi.org/10.1137/0316002Google Scholar
  32. R. T. Rockafellar, R. J.-B. Wets: Deterministic and stochastic optimization problems of Bolza type in discrete time, Stochastics 10/3-4 (1983) 273–312.https://doi.org/10.1080/17442508308833276Google Scholar
  33. R. T. Rockafellar, R. J.-B. Wets: Variational analysis, Grundlehren der Mathematischen Wissenschaften Vol. 317, Springer, Berlin (1998).https://doi.org/10.1007/978-3-642-02431-3Google Scholar
  34. A. N. Shiryaev: Probability, 2nd ed., Graduate Texts in Mathematics Vol. 95, Springer, New York (1996).https://doi.org/10.1007/978-1-4757-2539-1Google Scholar
  35. L. Thibault: Espérances conditionnelles d’intégrandes semi-continus, Ann. Inst. H. Poincaré Sect. B (N.S.) 17/4 (1981) 337–350.Google Scholar
  36. A. Truffert: Conditional expectation of integrands and random sets, Ann. Oper. Res. 30/1-4 (1991) 117–156.https://doi.org/10.1007/BF02204813Google Scholar
  37. R. J.-B. Wets: Programming under uncertainty: The complete problem, Z. Wahrsch. Verw. Gebiete 4 (1965/66) 316–339.https://doi.org/10.1007/BF00539117Google Scholar
  38. R. J.-B. Wets: Programming under uncertainty: The equivalent convex program, SIAM J. Appl. Math. 14 (1966) 89–105.https://doi.org/10.1137/0114008Google Scholar

Suggested citation

T. Pennanen, A.-P. Perkkiö. “Dynamic Programming in Convex Stochastic Optimization.” Journal of Convex Analysis 30 (2023), No. 4, 1241–1283. https://doi.org/10.68381/jca30056

Download BibTeXPublished by Heldermann Verlag.

Published by Heldermann Verlag, 2023. Rights now held by Banach Press LLC.