Kalman filtering (and filtering in general) considers the following setting: we have a sequence of states , which evolves under random perturbations over time. Unfortunately we cannot observe , we can only observe some noisy function of , namely, . Our task is to find the best estimate of given our observations of . Continue reading “Kalman Filter”

# Category: Control for Finance

## Sequential Investment

We consider the problem of sequentially investing in a set of stocks.

## Experts and Bandits (non-stochastic)

- Weighted majority algorithm its variant for Bandit Problems.

## More on Merton Portfolio Optimization

## Diffusion Control Problems

- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
- Davis-Varaiya Martingale Prinicple for Optimality

## Stochastic Integration – a Heuristic view

Heuristic derivation of

- the Stochastic Integral
- Stochastic Differential Equations
- Ito’s Formula

Continue reading “Stochastic Integration – a Heuristic view”

## Continuous Time Dynamic Programs

- Continuous-time dynamic programs
- The HJB equation; a heuristic derivation; and proof of optimality.

## Optimal Stopping

- Optimal Stopping Problems; One-Step-Look-Ahead Rule
- The Secretary Problem.
- Infinite Time Stopping

## Algorithms for MDPs

- High level idea: Policy Improvement and Policy Evaluation.
- Value Iteration; Policy Iteration.
- Temporal Differences; Q-factors.

## Infinite Time Horizon, MDP

- Positive Programming, Negative Programming & Discounted Programming.
- Optimality Conditions.