Here are the slides from Lectures

Please read Section 2.2 of the notes

Skip to content
## Stochastic Integration (a quick intro)

## Continuous Time Dynamic Programming

## LQR and Kalman Filter

## ODE method for Stochastic Approximation

## Lyapunov Functions

## Optimal Stopping

## Algorithms for MDPs

## Infinite Time Horizon MDP

## Markov Decision Processes

## Markov Chains

Here are the slides from Lectures

Please read Section 2.2 of the notes

Here are the slides from Lectures

8_Continuous Time Dynamic Programming

Please read Section 2.1 of the notes

Here are the slides from Lectures

Please read these notes [which will be later added to the main set of notes]:

We consider the Robbins-Monro update

and argue that this can be approximated by the o.d.e.:

Lyapunov functions are an extremely convenient device for proving that a dynamical system converges. We cover:

- The Lyapunov argument
- La Salle’s Invariance Principle
- An Alternative argument for Convex Functions
- Exponential Convergence Rates

Here are the slides from Lectures

Please read Section 1.6 from the notes:

Please attempt Ex53, Ex54, Ex56, Ex57.

Here are the slides from Lectures

Please read Section 1.5 from the notes:

Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.

Here are the slides from Lectures

Please read Section 1.4 from the notes:

Please attempt Ex35, Ex36, Ex37.

Here are the slides from Lectures

3_Markov Decision Processes [pdf]

Please read Section 1.3 from the notes:

Please attempt exercises Ex22, Ex23, Ex24, Ex25.

Here are the slides from Lectures

Please read Section 1.2 from the notes: