Discrete time Dynamic Programming was given in the post Dynamic Programming. We now consider the continuous time analogue.
Author: appliedprobability
Optimal Stopping
An Optimal Stopping Problem is an Markov Decision Process where there are two actions: 
This defines a stopping problem.
Algorithms for MDPs
For infinite time MDPs, we cannot apply to induction on Bellman’s equation from some initial state – like we could for finite time MDP. So we need some algorithms to solve MDPs.
Markov Chains: A Quick Review
This section is intended as a brief introductory recap of Markov chains. A much fuller explanation and introduction is provided in standard texts e.g. Norris, Bremaud, or Levin & Peres (see references below).
Infinite Time Horizon
Thus far we have considered finite time Markov decision processes. We now want to solve MDPs of the form 
Markov Decision Processes
Markov decision processes are essentially the randomized equivalent of a dynamic program.
Dynamic Programming
We briefly explain the principles behind dynamic programming and then give its definition.
Notes for Stochastic Control 2019
The link below contains notes PDF for this years stochastic control course
I’ll upload individual posts for each section. I’ll likely update these notes and add more exercises over the coming semester. I’ll add this update in a further post at the end of the course. Comments, typos, suggestions are always welcome.
Markov’s Inequality, Chebychev’s Inequality and the Weak Law of Large Numbers
Probability Son, Father and Grandfather have same birthday.
Here is a quick request for comments for Probability 1 students. Here are two answers saying that the probability that a grandfather, father and son are all born on the same day.
https://www.mirror.co.uk/news/uk-news/baby-boy-born-same-date-13416140
https://www.bbc.co.uk/news/world-us-canada-41406855#
The first answer is sort of wrong because it assumes you specify in advance date of birth. The second answer is right because we assume in advance we are given three generations and we assume we deal with a first born son.
Please leave comments below and will forward them on to BBC.
Applied Probability Notes 