We define a discrete-time queueing network where there are restrictions on which queues can be served simultaneously. We give a policy for serving queues which is stable whenever it is possible to stabilize the queueing network.

# Author: appliedprobability

## Blackwell Approachability

Sequentially a player decides to play and his adversary decides . At time , a decision results in a vector payoff . Given is the average vector payoff at time , Blackwell’s Approachability Theorem is a necessary and sufficient condition so that, regardless of the adversary’s decisions, the player makes the sequence of vectors approach a convex set .

## Weighted Majority Algorithm

The Weighted Majority Algorithm is a randomized rule used to learn the best action amongst a fixed reference set.

## Power-of-k Choices

We consider a model of a large number of single server queues. When a job arrives, we assume it chooses between queues. The job then chooses to join the shortest of these queues. We show that such choice can dramatically reduce queue sizes.

## Little’s Law

Here is the long run queue length; is the expected waiting time; is the arrival rate at the queue.

## Diffusion Control Problems

- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.

## Continuous Time Dynamic Programs

- Continuous-time dynamic programs
- The HJB equation; a heuristic derivation; and proof of optimality.

## Markov Decision Processes

- Markov Decisions Problems; Bellman’s Equation; Two examples

## Dynamic Programming

- Dynamic Programs; Bellman’s Equation; An example.

## Ito’s Formula: a heuristic derivation

- A heuristic look at the stochastic integral.
- heuristic derivation of Itô’s formula.