Control – Page 2 – Applied Probability Notes

Continuous Time Dynamic Programs

Continuous-time dynamic programs
The HJB equation; a heuristic derivation; and proof of optimality.

Continue reading “Continuous Time Dynamic Programs”

Algorithms for MDPs

High level idea: Policy Improvement and Policy Evaluation.
Value Iteration; Policy Iteration.
Temporal Differences; Q-factors.

Continue reading “Algorithms for MDPs”

Infinite Time Horizon, MDP

Positive Programming, Negative Programming & Discounted Programming.
Optimality Conditions.

Continue reading “Infinite Time Horizon, MDP”

Markov Decision Processes

Markov Decisions Problems; Bellman’s Equation; Two examples

Continue reading “Markov Decision Processes”

Dynamic Programming

Dynamic Programs; Bellman’s Equation; An example.

Continue reading “Dynamic Programming”

Lyapunov functions

Lyapunov functions are an extremely convenient device for proving that a dynamical system converges.

Continue reading “Lyapunov functions”

Sequentially a player decides to play $\{p_t\}_{t=1}^\infty$ and his adversary decides $\{q_t\}_{t=1}^\infty$ . At time $t$ , a decision $(p_t,q_t)$ results in a vector payoff $A(p_t,q_t)\in {\mathbb R}^k$ . Given $a_t$ is the average vector payoff at time $t$ , Blackwell’s Approachability Theorem is a necessary and sufficient condition so that, regardless of the adversary’s decisions, the player makes the sequence of vectors $\{a_t\}_{t=1}^\infty$ approach a convex set ${\mathcal A}$ .