Uncategorized – Page 2 – Applied Probability Notes

Diffusion Control

Here are the slides from Lectures

10_Diffusion Control

Please read Section 2.3 of the notes

Stochastic_Control_2020

Stochastic Integration (a quick intro)

Here are the slides from Lectures

9_Stochastic Integration

Please read Section 2.2 of the notes

Stochastic_Control_2020

Continuous Time Dynamic Programming

Here are the slides from Lectures

8_Continuous Time Dynamic Programming

Please read Section 2.1 of the notes

Stochastic_Control_2020

LQR and Kalman Filter

Here are the slides from Lectures

7_LQR_and_Kalman_Filter

Please read these notes [which will be later added to the main set of notes]:

LQR_PDF

ODE method for Stochastic Approximation

We consider the Robbins-Monro update

$x_{n+1} = x_n +\alpha_n [g(x_n) + \epsilon_n]$

and argue that this can be approximated by the o.d.e.:

$\frac{dz}{d t} = g(z(t)).$

Continue reading “ODE method for Stochastic Approximation”

Optimal Stopping

Here are the slides from Lectures

6_Optimal Stopping

Please read Section 1.6 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex53, Ex54, Ex56, Ex57.

Algorithms for MDPs

Here are the slides from Lectures

5_Algorithms for MDPs

Please read Section 1.5 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.

Infinite Time Horizon MDP

Here are the slides from Lectures

4_Infinite Time Horizon

Please read Section 1.4 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex35, Ex36, Ex37.

Markov Decision Processes

Here are the slides from Lectures

3_Markov Decision Processes [pdf]

Please read Section 1.3 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt exercises Ex22, Ex23, Ex24, Ex25.

Markov Chains

Here are the slides from Lectures

2_Markov_Chains

Please read Section 1.2 from the notes:

Stochastic_Control_2020