Kalman filtering (and filtering in general) considers the following setting: we have a sequence of states , which evolves under random perturbations over time. Unfortunately we cannot observe , we can only observe some noisy function of , namely, . Our task is to find the best estimate of given our observations of . Continue reading “Kalman Filter”
We consider the problem of sequentially investing in a set of stocks.
- Weighted majority algorithm its variant for Bandit Problems.
- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
- Davis-Varaiya Martingale Prinicple for Optimality
Heuristic derivation of
- the Stochastic Integral
- Stochastic Differential Equations
- Ito’s Formula
- Continuous-time dynamic programs
- The HJB equation; a heuristic derivation; and proof of optimality.
- Optimal Stopping Problems; One-Step-Look-Ahead Rule
- The Secretary Problem.
- Infinite Time Stopping
- High level idea: Policy Improvement and Policy Evaluation.
- Value Iteration; Policy Iteration.
- Temporal Differences; Q-factors.
- Positive Programming, Negative Programming & Discounted Programming.
- Optimality Conditions.