Uncategorized – Page 3 – Applied Probability Notes

Markov Chains

Here are the slides from Lectures

2_Markov_Chains

Please read Section 1.2 from the notes:

Stochastic_Control_2020

Dynamic Programming

Slides from Lectures are here:

1_Dynamic_Programming

Please read Section 1.1 of the notes:

Stochastic Control Notes [pdf]

Please attempt exercises Ex3, Ex5 and Ex6 from the notes.

Stochastic Control 2020

Another year of MATH69122! — aka Stochastic Control.

This year, I will try to keep updating PDFs with slides and notes for each lecture. I’ll keep notes for the course in the “PDF” tab above. These are also here:

Stochastic Control 2020 [pdf]

Here is a rough plan for each week of lectures:

Continue reading “Stochastic Control 2020”

Kalman filtering (and filtering in general) considers the following setting: we have a sequence of states $x_t$ , which evolves under random perturbations over time. Unfortunately we cannot observe $x_t$ , we can only observe some noisy function of $x_t$ , namely, $y_t$ . Our task is to find the best estimate of $x_t$ given our observations of $y_t$ . Continue reading “Kalman Filter”

Mixing Times

In loose terms, the mixing time is the amount of time to wait before you can expect a Markov chain to be close to its stationary distribution. We give an upper bound for this.

Continue reading “Mixing Times”

Diffusion Control Problems

We consider a continuous time analogue of Markov Decision Processes.

Continue reading “Diffusion Control Problems”

Notes for Stochastic Control 2019

The link below contains notes PDF for this years stochastic control course

stochastic_control_2019

I’ll upload individual posts for each section. I’ll likely update these notes and add more exercises over the coming semester. I’ll add this update in a further post at the end of the course. Comments, typos, suggestions are always welcome.

Markov’s Inequality, Chebychev’s Inequality and the Weak Law of Large Numbers

Probability Son, Father and Grandfather have same birthday.

Here is a quick request for comments for Probability 1 students. Here are two answers saying that the probability that a grandfather, father and son are all born on the same day.

https://www.mirror.co.uk/news/uk-news/baby-boy-born-same-date-13416140

https://www.bbc.co.uk/news/world-us-canada-41406855#

The first answer is sort of wrong because it assumes you specify in advance date of birth. The second answer is right because we assume in advance we are given three generations and we assume we deal with a first born son.

Please leave comments below and will forward them on to BBC.

Category: Uncategorized