appliedprobability

Stochastic Integration (a quick intro)

Here are the slides from Lectures

9_Stochastic Integration

Please read Section 2.2 of the notes

Stochastic_Control_2020

Continuous Time Dynamic Programming

Here are the slides from Lectures

8_Continuous Time Dynamic Programming

Please read Section 2.1 of the notes

Stochastic_Control_2020

LQR and Kalman Filter

Here are the slides from Lectures

7_LQR_and_Kalman_Filter

Please read these notes [which will be later added to the main set of notes]:

LQR_PDF

ODE method for Stochastic Approximation

We consider the Robbins-Monro update

$x_{n+1} = x_n +\alpha_n [g(x_n) + \epsilon_n]$

and argue that this can be approximated by the o.d.e.:

$\frac{dz}{d t} = g(z(t)).$

Continue reading “ODE method for Stochastic Approximation”

Lyapunov Functions

Lyapunov functions are an extremely convenient device for proving that a dynamical system converges. We cover:

The Lyapunov argument
La Salle’s Invariance Principle
An Alternative argument for Convex Functions
Exponential Convergence Rates

Continue reading “Lyapunov Functions”

Optimal Stopping

Here are the slides from Lectures

6_Optimal Stopping

Please read Section 1.6 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex53, Ex54, Ex56, Ex57.

Algorithms for MDPs

Here are the slides from Lectures

5_Algorithms for MDPs

Please read Section 1.5 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex39, 40 & 41 [if you can code], 42 and 43.

Infinite Time Horizon MDP

Here are the slides from Lectures

4_Infinite Time Horizon

Please read Section 1.4 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt Ex35, Ex36, Ex37.

Markov Decision Processes

Here are the slides from Lectures

3_Markov Decision Processes [pdf]

Please read Section 1.3 from the notes:

Stochastic_Control_2020 [pdf]

Please attempt exercises Ex22, Ex23, Ex24, Ex25.

Markov Chains

Here are the slides from Lectures

2_Markov_Chains

Please read Section 1.2 from the notes:

Stochastic_Control_2020