Stochastic control theory 1000-1M23TOS
The course is devoted to the survey of basic tools of the stochastic control theory, the considerations will be illustrated by numerous examples and applications. Most of the material will be discussed in the context of discrete-time processes. In particular, the presentation will cover the maximum principle, the Hamilton-Jacobi-Bellman equation and dynamic programming.
1. Introduction. Selected examples in the deterministic control theory. (2 lectures)
2. Dynamic programming, examples (2 lectures).
3. Maximum principle. The Hamilton-Jacobi-Bellman equation (3 lectures).
4. A distinguished case: optimal stopping theory (4 lectures).
5. Elements of optimal control theory for continuous-time processes (3-4 lectures).
Main fields of studies for MISMaP
Type of course
Mode
Requirements
Prerequisites
Prerequisites (description)
Course coordinators
Learning outcomes
Knowledge and skills. A Student:
1. Gives examples of deterministic control problems and formulates the general methods of the investigation.
2. Knows the concept of dynamic programming and applies it in the study of the problems of optimal control theory.
3. Formulates the maximum principle and knows its connections to the Hamilton-Jacobi-Bellman equation.
4. Formulates and solves basic problems in optimal stopping theory, both for the finite and infinite horizon.
5. Knows basic facts concerning the optimal control theory for continuous-time processes.
6. Knows the up-to-date achievements of the theory, enabling the individual research in the area.
Social competence. A Student
1. Understands the role of the control theory as a tool for the investigation of certain mechanisms of Nature
Assessment criteria
Two written homework assignments during the semester and a final oral exam.
Bibliography
1. P. D. Bertsekas, S. E. Shreve, Stochastic optimal control. The discrete time case. Mathematics in Science and Engineering, 139. Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New York-London, 1978.
2. A. Seierstad, Stochastic control in discrete and continuous time. Springer, New York, 2009.
3. Up-to-date lecture notes will be available at https://www.mimuw.edu.pl/~ados/teaching/index.html
Additional information
Information on level of this course, year of study and semester when the course unit is delivered, types and amount of class hours - can be found in course structure diagrams of apropriate study programmes. This course is related to the following study programmes:
Additional information (registration calendar, class conductors, localization and schedules of classes), might be available in the USOSweb system: