Here are the slides from Lectures

Please read Section 2.3 of the notes

Skip to content
## Diffusion Control

## Stochastic Integration (a quick intro)

## Continuous Time Dynamic Programming

## LQR and Kalman Filter

## ODE method for Stochastic Approximation

## Lyapunov Functions

## Optimal Stopping

## Algorithms for MDPs

## Infinite Time Horizon MDP

## Markov Decision Processes

Here are the slides from Lectures

Please read Section 2.3 of the notes

Here are the slides from Lectures

Please read Section 2.2 of the notes

Here are the slides from Lectures

8_Continuous Time Dynamic Programming

Please read Section 2.1 of the notes

Here are the slides from Lectures

Please read these notes [which will be later added to the main set of notes]:

We consider the Robbins-Monro update

and argue that this can be approximated by the o.d.e.:

Lyapunov functions are an extremely convenient device for proving that a dynamical system converges. We cover:

- The Lyapunov argument
- La Salle’s Invariance Principle
- An Alternative argument for Convex Functions
- Exponential Convergence Rates

Here are the slides from Lectures

Please read Section 1.6 from the notes:

Please attempt Ex53, Ex54, Ex56, Ex57.

Here are the slides from Lectures

Please read Section 1.5 from the notes:

Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.

Here are the slides from Lectures

Please read Section 1.4 from the notes:

Please attempt Ex35, Ex36, Ex37.

Here are the slides from Lectures

3_Markov Decision Processes [pdf]

Please read Section 1.3 from the notes:

Please attempt exercises Ex22, Ex23, Ex24, Ex25.