We consider a network of wireless routers. The routers that are close together can interfere if they transmit simultaneously. So schedules need to avoid such collisions. We want each each wireless node to achieve a transmission rates that equals its arrival rate. One might want to implement a policy like MaxWeight or simply estimate the vector of arrival rates and accordingly choose the correct transmission rate. However, this is complicated by the fact that the routers do not know the arrival and transmission rates of their neighbors; all they can do is sense if their neighbors are transmitting or not.
Max-Weight Scheduling
We define a discrete-time queueing network where there are restrictions on which queues can be served simultaneously. We give a policy for serving queues which is stable whenever it is possible to stabilize the queueing network.
Blackwell Approachability
Sequentially a player decides to play and his adversary decides
. At time
, a decision
results in a vector payoff
. Given
is the average vector payoff at time
, Blackwell’s Approachability Theorem is a necessary and sufficient condition so that, regardless of the adversary’s decisions, the player makes the sequence of vectors
approach a convex set
.
Weighted Majority Algorithm
The Weighted Majority Algorithm is a randomized rule used to learn the best action amongst a fixed reference set.
Power-of-k Choices
We consider a model of a large number of single server queues. When a job arrives, we assume it chooses between queues. The job then chooses to join the shortest of these queues. We show that such choice can dramatically reduce queue sizes.
Little’s Law
Here is the long run queue length;
is the expected waiting time;
is the arrival rate at the queue.
Diffusion Control Problems
- The Hamilton-Jacobi-Bellman Equation.
- Heuristic derivation of the HJB equation.
Continuous Time Dynamic Programs
- Continuous-time dynamic programs
- The HJB equation; a heuristic derivation; and proof of optimality.
Markov Decision Processes
- Markov Decisions Problems; Bellman’s Equation; Two examples
Dynamic Programming
- Dynamic Programs; Bellman’s Equation; An example.