Here are the slides from Lectures
Please read Section 2.3 of the notes
Stochastic Integration (a quick intro)
Here are the slides from Lectures
Please read Section 2.2 of the notes
Continuous Time Dynamic Programming
Here are the slides from Lectures
8_Continuous Time Dynamic Programming
Please read Section 2.1 of the notes
LQR and Kalman Filter
Here are the slides from Lectures
Please read these notes [which will be later added to the main set of notes]:
ODE method for Stochastic Approximation
We consider the Robbins-Monro update
and argue that this can be approximated by the o.d.e.:
Lyapunov Functions
Lyapunov functions are an extremely convenient device for proving that a dynamical system converges. We cover:
- The Lyapunov argument
- La Salle’s Invariance Principle
- An Alternative argument for Convex Functions
- Exponential Convergence Rates
Optimal Stopping
Here are the slides from Lectures
Please read Section 1.6 from the notes:
Please attempt Ex53, Ex54, Ex56, Ex57.
Algorithms for MDPs
Here are the slides from Lectures
Please read Section 1.5 from the notes:
Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.
Infinite Time Horizon MDP
Here are the slides from Lectures
Please read Section 1.4 from the notes:
Please attempt Ex35, Ex36, Ex37.
Markov Decision Processes
Here are the slides from Lectures
3_Markov Decision Processes [pdf]
Please read Section 1.3 from the notes:
Please attempt exercises Ex22, Ex23, Ex24, Ex25.
