We consider a specific diffusion control problem. We focus on setting where there is one risky asset and one riskless asset, though we will see that much of the analysis passes over to multiple assets.

# Category: MATH69122 Stochastic Control for Finance

## Diffusion Control Problems

- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
- Davis-Varaiya Martingale Prinicple for Optimality

## Stochastic Integration – a Heuristic view

Heuristic derivation of

- the Stochastic Integral
- Stochastic Differential Equations
- Ito’s Formula

Continue reading “Stochastic Integration – a Heuristic view”

## Continuous Time Dynamic Programs

- Â Continuous-time dynamic programs
- The HJB equation; a heuristic derivation; and proof of optimality.

## Optimal Stopping

- Optimal Stopping Problems; One-Step-Look-Ahead Rule
- The Secretary Problem.
- Infinite Time Stopping

## Algorithms for MDPs

- High level idea: Policy Improvement and Policy Evaluation.
- Value Iteration; Policy Iteration.
- Temporal Differences; Q-factors.

## Infinite Time Horizon, MDP

- Positive Programming, Negative Programming & Discounted Programming.
- Optimality Conditions.

## Markov Decision Processes

- Markov Decisions Problems; Bellman’s Equation; Two examples

## Markov Chains

- A short introduction to Markov chains for dynamic programming
- Definition, Markov Property, some Potential Theory.