Here are the slides from Lectures
Please read Section 1.5 from the notes:
Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.
Category: Uncategorized
Infinite Time Horizon MDP
Here are the slides from Lectures
Please read Section 1.4 from the notes:
Please attempt Ex35, Ex36, Ex37.
Markov Decision Processes
Here are the slides from Lectures
3_Markov Decision Processes [pdf]
Please read Section 1.3 from the notes:
Please attempt exercises Ex22, Ex23, Ex24, Ex25.
Markov Chains
Here are the slides from Lectures
Please read Section 1.2 from the notes:
Dynamic Programming
Slides from Lectures are here:
Please read Section 1.1 of the notes:
Stochastic Control Notes [pdf]
Please attempt exercises Ex3, Ex5 and Ex6 from the notes.
Stochastic Control 2020
Another year of MATH69122! — aka Stochastic Control.
This year, I will try to keep updating PDFs with slides and notes for each lecture. I’ll keep notes for the course in the “PDF” tab above. These are also here:
Here is a rough plan for each week of lectures:
Kalman Filter
Kalman filtering (and filtering in general) considers the following setting: we have a sequence of states 
Mixing Times
In loose terms, the mixing time is the amount of time to wait before you can expect a Markov chain to be close to its stationary distribution. We give an upper bound for this.
Diffusion Control Problems
We consider a continuous time analogue of Markov Decision Processes.
Notes for Stochastic Control 2019
The link below contains notes PDF for this years stochastic control course
I’ll upload individual posts for each section. I’ll likely update these notes and add more exercises over the coming semester. I’ll add this update in a further post at the end of the course. Comments, typos, suggestions are always welcome.
