Applied Probability Notes

Algorithms for MDPs

Here are the slides from Lectures

5_Algorithms for MDPs

Please read Section 1.5 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.

Infinite Time Horizon MDP

Here are the slides from Lectures

4_Infinite Time Horizon

Please read Section 1.4 from the notes:

Stochastic_Control_2020 [pdf]

Markov Decision Processes

Here are the slides from Lectures

3_Markov Decision Processes [pdf]

Please read Section 1.3 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt exercises Ex22, Ex23, Ex24, Ex25.

Markov Chains

Here are the slides from Lectures

2_Markov_Chains

Please read Section 1.2 from the notes:

Stochastic_Control_2020

Dynamic Programming

Slides from Lectures are here:

1_Dynamic_Programming

Please read Section 1.1 of the notes:

Stochastic Control Notes [pdf]

Please attempt exercises Ex3, Ex5 and Ex6 from the notes.

Stochastic Control 2020

Another year of MATH69122! — aka Stochastic Control.

This year, I will try to keep updating PDFs with slides and notes for each lecture. I’ll keep notes for the course in the “PDF” tab above. These are also here:

Stochastic Control 2020 [pdf]

Here is a rough plan for each week of lectures:

Continue reading “Stochastic Control 2020”

Kalman Filter

Kalman filtering (and filtering in general) considers the following setting: we have a sequence of states $x_t$ , which evolves under random perturbations over time. Unfortunately we cannot observe $x_t$ , we can only observe some noisy function of $x_t$ , namely, $y_t$ . Our task is to find the best estimate of $x_t$ given our observations of $y_t$ . Continue reading “Kalman Filter”