What follows is a heuristic derivation of the Stochastic Integral, Stochastic Differential Equations and Itô’s Formula.
Author: appliedprobability
Continuous Time Dynamic Programming
Discrete time Dynamic Programming was given in the post Dynamic Programming. We now consider the continuous time analogue.
Optimal Stopping
An Optimal Stopping Problem is an Markov Decision Process where there are two actions: 
This defines a stopping problem.
Algorithms for MDPs
For infinite time MDPs, we cannot apply to induction on Bellman’s equation from some initial state – like we could for finite time MDP. So we need some algorithms to solve MDPs.
Markov Chains: A Quick Review
This section is intended as a brief introductory recap of Markov chains. A much fuller explanation and introduction is provided in standard texts e.g. Norris, Bremaud, or Levin & Peres (see references below).
Infinite Time Horizon
Thus far we have considered finite time Markov decision processes. We now want to solve MDPs of the form 
Markov Decision Processes
Markov decision processes are essentially the randomized equivalent of a dynamic program.
