Sequential Investment

We consider the problem of sequentially investing in a set of stocks.

However to start with we begin with the example of gambling on horses.

Consider a sequence of $T$ horse races.
Each race has $N$ horses and only one hose can win.
Let $\mathbf h_t$ be the unit vector indicating the winning horse in the $t$ th race i.e. $\mathbf h_t=(0,...,0,1,0,...,0)\in \{0,1\}^N$ with the one in the position of the $t$ th winning horse. Further let $\mathbf H_t = (\mathbf h_\tau : \tau =1,...,t).$
Let ${p}_t(i | \mathbf H_{t-1})$ be the proportion of our wealth that we invest in horse $i$ given the winners so far $\mathbf H_{t-1}$ .
If horse $n$ wins we multiply our wealth by $m_t(i)>0$ and $m_t(j)=0$ for all $j\neq i$ .
We wish to compare the performance of our prediction policy for betting $p_t(\cdot | \mathbf H_{t-1})$ and some set of experts $q_t(\cdot | \mathbf H_{t-1}) \in \mathcal Q$ .

We want to compare the performance of our policy with the best expert. First we show that we can express this in a regret framework.

Ex 1. Argue that after $T$ races, the ratio between our wealth and the best expert in $\mathcal Q$ is given by

$\sup_{q\in\mathcal Q} \frac{q_T(\bm H_T)}{p_T(\bm H_T)} \qquad\qquad \text{where} \qquad p_T(\bm H_T) = \prod_{t=1}^T p_t(\bm h_t | \bm H_t )$

and similarly for $q_T(\mathbf H_T)$ .

Ans 1. Starting with an initial wealth of $C$ , the wealth attained from following $p$ and $q$ respectively are

$C \prod_{t=1}^T p_t(\bm h_t | \bm H_{t-1} ) m_t(\bm h_t) \qquad \text{and} \qquad C \prod_{t=1}^T q_t(\bm h_t | \bm H_{t-1} ) m_t(\bm h_t)$

Now divide the second term by the first and maximize over $q\in\mathcal Q$ .

Ex 2. [Continued] Argue that the above ratio can be expressed in terms of regret:

$\mathcal R\! g_T(p ; \bm H_T) = L_T(p) - \min_{q\in \mathcal Q} L_T(q)$

where the loss of a probability distribution $p =(p(i) : i=1,...,N)$ and horse $h$ winning is given by

$l(p,h) = \log \frac{1}{p(h)}$

Ans 2. Take logs in [1] $\max_{q\in\mathcal Q} \log \left( \frac{q_T(\bm H_T)}{p_T(\bm H_T)} \right) = \log \frac{1}{p_T(\bm H_T)} - \min_{q\in \mathcal Q} \log \frac{1}{q_T(\bm H_T)}$ and

$\log \frac{1}{p_T(\bm H_T)} = \sum_{t=1}^T \log \frac{1}{p_t(\bm h_t | \bm H_{t-1})} = \sum_{t=1}^T l(p_t | \bm h_t) = L_T(p_T)\, .$

Def 3. [Minimax Regret] We look at the worse regret that a specific strategy $p$ can have relative to experts $\mathcal Q$ ,

$\begin{aligned} \mathcal V_T(p,\mathcal Q) &= \max_{\bm H_T} \mathcal R\! g_T(p , \mathcal Q; \bm H_T) \\ &= \max_{\bm H_T} \max_{q_T\in \mathcal Q} \log \left( \frac{q_T(\bm H_T)}{p_T(\bm H_T)} \right) \end{aligned}$

and then we look for the best strategy

$\mathcal V_T(\mathcal Q) = \min_{p} \mathcal V_T( p,\mathcal Q)$

Here $\mathcal V_T(\mathcal Q)$ is called the minimax regret.

As we will see there is a unique policy that achieves this regret.

Def 4. [Normalized Maximum Likelihood Forcaster] This is the policy defined by

$p^*_T(\bm H_T) = \frac{ \max_{q \in \mathcal Q} q_T(\bm H_T) }{ \sum_{\bm H'_T} \max_{q \in \mathcal Q} q_T(\bm H'_T) }$

You can bet a proportion wealth

where there is only a prize for winning

Ex 5. For any class of experts $\mathcal Q$ and integrer $T$ , the normalized maximum likelihood forcaster is the unique forecaster such that

$\mathcal V_T(p^*,\mathcal Q) = \mathcal V_T( \mathcal Q)$

Ans 5. For a distribution $p \neq p^*$ $\exists \mathbf H_T$ such that $p_T(\mathbf H_T) < p^*(\mathbf H_T)$ (remembering both distributions sum to one). So,

$\begin{aligned} \mathcal V_T( p ,\mathcal Q) \geq \log \left( \frac{ \max_{q\in \mathcal Q} q_T(\mathcal Q) }{ p_T(\bm H_T ) } \right) & \geq \log \left( \frac{ \max_{q\in \mathcal Q} q_T(\mathcal Q) }{ p^*_T(\bm H_T ) } \right) \\ & = \log \left( \sum_{\bm H'_T} \sup_{q\in\mathcal Q} q_T(\bm H'_T ) \right) =\mathcal V_T( p^* ,\mathcal Q) \, .\end{aligned}$

So $\min_p \mathcal V_T(p,\mathcal Q) \geq \mathcal V_T(p^*,\mathcal Q)$ .

Ex 6. [Continued] Show that for all $\mathbf H_T$

$\mathcal V_T( \mathcal Q) = \log \left( \frac{ \sup_{q\in \mathcal Q} q_T(\bm H_T) }{ p^*_T(\bm H_T) } \right) = \log \left( \sum_{\bm H'_T} \sup_{q\in\mathcal Q} q_T(\bm H'_T ) \right)\, .$

Ans 6. This follows from the answer to [5].

Sequential Investment

We now develop put theory for investment strategies. Here there is now longer one winning horse/stock. We consider the goal of maximizing final wealth with respect to a set of reference strategies.

An investor trades over $T$ days.
We consider a market with $N$ stocks.
A market vector $\mathbf S = (S(1),...,S(N))$ gives the (relative) prices of stocks over each period given period.
For a sequence of market vectors $\mathbf S_\tau$ , $\tau=1,...,t$ , we let $\mathcal H_t = (\mathbf S_\tau : \tau=1,..,t )$ denote the history of market vectors before time $t$ .
An investment is a probability distribution $\mathbf \pi = (\pi(1),...,\pi(N)) \in \mathcal D$ that give the proportions of (one unit of) wealth invested in each asset. This results in an increase in wealth of
$\bm S^\top \bm Q =\sum_{i=1}^N S(i) Q_i\, .$
An investment strategy, ${\Pi}$ , is a sequence of investments $\mathbf \pi_t (\mathcal H_{t-1}) = ( \pi_t ( i | \mathcal H_{t-1}) : i=1,...,N)$ for each time $t=1,..,T$ .

Def 7. [Wealth Factor] The wealth factor of an on investment strategy is given by

$\mathcal W\! f_T({\Pi},\mathcal H_T) = \prod_{t=1}^T \big\{ \bm S_{t}^\top \bm \pi_t (\mathcal H_{t-1}) \big\}$

Def 8. [Market Vector] For a vector ${\mathbf S}=(S^1,...,S^d)$ giving the relative prices of $d$ stocks, a Market vector investment in each of these stocks ${\mathbf n}=(n^1,...,n^d)$ and this results in an increase in wealth

${\mathbf n}\cdot {\mathbf S} = \sum_{i=1}^d n^i S^i.$

Def 9. [Wealth Factor] For a sequence of investments and stock prices ${\mathbf n}^T\! =({\mathbf n}_t : t=1,...,T)$ and $\mathbf{S}^T\! =({\mathbf S}_t : t=1,...,T)$ , the wealth factor is the increase in wealth from time $t=1,..,T$

$W\! f({\mathbf n}^T,{\mathbf S}^T) = \prod_{t=1}^T \left( {\mathbf n}_t\cdot {\mathbf S}_t \right)$

Def 10. [Investment Strategy] An investment strategy $\Pi$ is a sequence of functions $\mathbf{\pi}_t({\mathbf S}^{t-1})$ , $t=1,...,T$ each giving a market vector.

Ex 11. [Buy-and-Hold] Buy-and-hold, $B h$ , is an investment strategy where you distribute you initial wealth according to some fixed proportion $Q(i)$ , $i=1,..,N$ and afterwards you don’t trade. Show that here the wealth factor is

$\mathcal W\! f_T( Bh, \mathcal H_T) = \sum_{i=1 }^N Q(i) \prod_{t=1}^T S_t(i)\, .$

Ans 11. It is clear the increase in asset $i$ is $\prod_{t=1}^T S_t(i)$ so the average increase is as given.

Ex 12. [Constantly rebalanced portfolio] A constantly rebalanced portfolio, $Cr$ , is a policy that keeps a fixed proportion of wealth, $\mathbf F =(F(i): i=1,...,N)$ , invested in each asset. Show that here the wealth factor is

$\mathcal W\! f_T( Cr, \mathcal H_T) = \prod_{t=1}^T \left( \bm F^\top \bm S_t \right)$

Ans 12. Now the increase in wealth at time $t$ is

$\sum_i F(i) S_t(i).$

Thus the total increase is the product of these terms.

Def 13. [Minimax Regret] For a class of investment strategies ${\mathcal I}$ , the regret of an investment strategy $\Pi$ is (similar to [2])

$\begin{aligned} \mathcal R\!g_T( \Pi, \mathcal P; \mathcal H_T ) =\log \left(\max_{P\in \mathcal P} \mathcal W\! f ( P , {\mathcal H}_T)\right) - \log \mathcal W\! f ( \Pi , {\mathcal H}_T)\, .\end{aligned}$

The minimax regret is then

$\mathcal W_T(\mathcal I) = \min_{\bm \Pi}\max_{{\mathcal H}_T\in \mathbb{R}_+^T} \mathcal R\!g_T(\Pi, \mathcal P)$

Ex 14. [Interpreting the minimax wealth ratio] Show that if $\mathcal R\! g_T (\Pi,{\mathcal P} )=F_T$ then $\Pi$ has the same growth rate as the best investment strategy in ${\mathcal P}$ , i.e. for all ${\mathbf S}^T$

$\max_{P \in {\mathcal P}} W\! f(P,{\mathbf H}_T) \leq W\! f(\Pi,{\mathbf H}_T) e^{F_T}.$

(From this exercise we can interpret the regret as the factor from which we are away from the best expert.)

Ex 15. Show that if

$\mathcal W_T(\mathcal P) \leq F_T$

then there exists a policy $\Pi$ such that for all histories $\mathcal H_T$

$\max_{\Pi\in \mathcal P}W\! f(\Pi,{\mathbf H}_T) \leq W\! f(\Pi,{\mathbf H}_T) e^{F_T}.$

(i.e. The wealth factor of the policu $\Pi$ is guarenteed to be within a factor $F_T$ of the best policy in $\mathcal P$ .)

Ans 15.

$\begin{aligned} \min_{\bm \Pi}\max_{{\mathbf H}_T\in \mathbb{R}_+^T} \mathcal R\!g_T(\Pi, \mathcal P) =:\mathcal W_T(\mathcal P) \leq F_T & \implies \exists\, \Pi \text{ s.t. } \qquad \max_{{\mathbf H}_T\in \mathbb{R}_+^T} \mathcal R\!g_T(\Pi, \mathcal P)\leq F_T \\ \implies & \exists \, \Pi \text{ s.t. } \forall \mathcal H_T, \qquad \mathcal R\!g_T(\Pi, \mathcal P)\leq F_T \\ \implies \exists \, \Pi \text{ s.t. }& \forall \mathcal H_T, \qquad\max_{P\in \mathcal P}W\! f(\Pi,{\mathbf H}_T) \leq W\! f(\Pi,{\mathbf H}_T) e^{F_T}. \end{aligned}$

The minimax wealth ratio is the best possible logarithmic wealth ratio $\begin{aligned} \mathcal{ W }_T ( \bm \Pi , {\mathcal P}) & = \max_{{\mathbf H}_T\in \mathbb{R}_+^T} \max_{P \in {\mathcal P} } \;\log \left( \frac{ \mathcal W\! f ( P , {\mathbf H}_T) }{ \mathcal W\! f (\bm \Pi , {\mathbf H}_T) } \right) \\ & = \mathcal R\!g_T(\Pi, \mathcal P) \end{aligned}$

$\mathcal W_T({\mathcal P} ) = \min_{\bm \Pi} \mathcal W_T(\bm \Pi, {\mathcal P}).$

Def 16. [Kelly Market Vector] A market vector that has only one non-zero component is called a Kelly Market vector, i.e. $\mathbf h_t=(0,...,0,1,0,...,0)\in \{0,1\}^N$ .

Note that we considered exactly Kelly market vectors in the previous horse race example.

We want to consider the differences for the “investment policies“ and “prediction policies” for the (horse) betting problem considered previously.

For every investment policy we can define a prediction policy:

Ex 17. Given an investment policy $\Pi$ we can define a prediction policy induced by $\Pi$ by

$p_t(i | \bm H_{t-1}) = \pi_t( i | \bm H_{t-1})$

Similarly, for every prediction policy we can define an investment policy:

Ans 17. Given a prediction strategy $p$ (as considered in the horse racing example), we can define an investment strategy induced by $\Pi$ by $\pi_{t} (i | \mathcal H_{t-1}) \propto \sum_{\bm H_{t-1} \in \{0,1\}^N} p_t ( j | \bm H_{t-1} ) p_{t-1}(\bm H_{t-1}) \prod_{\tau=1}^{t-1}S_{\tau}(\bm h_{\tau})$

Nb, note that the normalization constant is given by

$F_t = \sum_{\bm H_t} p_t(\bm H_{t}) \prod_{\tau=1}^{t-1} S_{\tau}(\bm h_{\tau})$

Ex 18. The wealth factor $\Pi$ the investment strategy induced prediction policy $p$ is

$\mathcal W\! f_T(\Pi, \mathcal H_t ) = F_{t+1} := \sum_{\bm H_{t+1}} p_t(\bm H_{t}) \prod_{\tau=1}^{t-1} S_{\tau}(\bm h_{\tau})$

where $F_t$ is given as above in Def. [SI:PinduceI].

Ans 18. From Def. 7, $\mathcal W\! f_T (\Pi, \mathcal H_t) = \prod_{t=1}^T \big\{ \mathbf S_{t}^\top \mathbf \pi_t (\mathcal H_{t-1}) \big\}$ . Looking at each term in this product

$\begin{aligned} \bm S_t^\top \bm \pi_t(\mathcal H_{t-1}) & = \sum_{\bm h_t} S_t(\bm h_t)\pi_t( \bm h_t | \mathcal H_{t-1} ) \\ & = \frac{1}{F_t} \sum_{\bm h_t} S_t(\bm h_t) \sum_{\bm H_{t-1} \in \{0,1\}^N} p_t ( j | \bm H_{t-1} ) p_{t-1}(\bm H_{t-1}) \prod_{\tau=1}^{t-1}S_{\tau}(\bm h_{\tau}) \\ & = \frac{1}{F_t} \sum_{\bm H_{t} \in \{0,1\}^N} p_{t}(\bm H_{t}) \prod_{\tau=1}^{t}S_{\tau}(\bm h_{\tau}) \\ & = \frac{ F_{t+1} }{ F_t }\, .\end{aligned}$ Thus

$\mathcal W\! f_T (\Pi, \mathcal H_t) = \prod_{t=1}^T \left( \frac{F_{t+1}}{F_t} \right) = F_{t+1}\, .$

In prinicple the investment problem should be more difficult that the prediction problem (used in horse racing). But the result below shows that actually they are in some sense equivalent.

Thrm 19. For a set of constantly rebalanced portfolios $\mathcal I$ and for the class of Prediction strategies $\mathcal P$ induced by $\mathcal I$ (cf. Def 17), we have that

$\mathcal W_T(\mathcal I) = \mathcal V_T(\mathcal P)\, .$

Moreover, in this case, the minimax optimal investment strategy is the investment strategy induced by the normalized maximum likelihood prediction policy.

The first example confirms the intuition that investing is harder than prediction:

Ex 20. [Continued – from Thrm 19] Show that

$\mathcal W_T(\mathcal I ) \geq \mathcal V_T(\mathcal P)$

Ans 20.

$\begin{aligned} \mathcal W_T (\mathcal I) = \max_{\mathcal H_t} \max_{\tilde{\Pi} \in \mathcal I} \log \left( \frac{ \mathcal W\! f ( \tilde{\Pi} , \mathcal H_T ) }{ \mathcal W\! f ( {\Pi} , \mathcal H_T ) } \right) & \geq \max_{\bm H_t \in \{ 0, 1\}^{N\cdot T}} \max_{\tilde{\Pi} \in \mathcal I} \log \left( \frac{ \mathcal W\! f ( \tilde{\Pi} , \bm H_T ) }{ \mathcal W\! f ( {\Pi} , \bm H_T ) } \right) \\ & = \max_{\bm H_t \in \{ 0, 1\}^{N\cdot T}} \max_{\tilde{\Pi} \in \mathcal P} \log \left( \frac{\tilde{\pi}(\bm H_T) }{ {\pi}(\bm H_T) } \right) \\ & = \mathcal V_T(p,\mathcal P) \geq \mathcal V_T (\mathcal P)\end{aligned}$

The first inequality holds since the Kelly market vectors are a subset of the set of Market Vectors. The subsequent equality holds since

$\mathcal W\! f (\Pi, \bm H_T) = \prod_{t=1}^T \big( S_t(\bm h_t) \pi_t(\bm h_t | \bm H_{t-1}) \big) = \bm \pi_T (\bm H_T) \prod_{t=1}^T S_t(\bm h_t)$

and we can replace policies in $\mathcal I$ and $\mathcal P$ since then coincide on Kelly market vectors (cf. Def 17). The final equality and inequality follow by the definition of the minimax regret in the horse racing example (Def. 3).

Ex 21. Show that for non-negative numbers $a_1,...,a_T$ and $b_1,...,b_T$

$\frac{ \sum_{t=1}^T a_t }{ \sum_{t=1}^T b_t } \leq \max_{t=1,...,T} \frac{a_t}{b_t} \, .$

Ans 21.

$\frac{ \sum_{t=1}^T a_t }{ \sum_{t=1}^T b_t } = \frac{ \sum_t \frac{a_t}{b_t} b_t }{ \sum_t b_t } = \frac{ \sum_t \max_{\tau} \frac{a_\tau}{b_\tau} b_t }{ \sum_t b_t } = \max_{\tau} \frac{a_\tau}{b_\tau}\, .$

Ex 22. Show that the wealth factor for any investment policy can be expressed as a sum over the Kelly market vectors

$\mathcal W\! f_T( \Pi , \mathcal H_T) = \sum_{\bm H_T} \prod_{t=1}^T S_t(\bm h_t) \pi_t(\bm h_t | \mathcal H_{t-1})$

Ans 22.

$\begin{aligned} \mathcal W\! f_T (\Pi , \mathcal H_T) & = \prod_{t=1}^T \left( \sum_{i=1}^N S_t(i) \pi_t(i | \mathcal H_{t-1}) \right) = \prod_{t=1}^T \left( \sum_{\bm h_t} S_t(\bm h_t) \pi_t(\bm h_t | \mathcal H_{t-1}) \right) \\ & = \sum_{\bm H_T} \prod_{t=1}^T S_t(\bm h_t) \pi_t(\bm h_t | \mathcal H_{t-1})\, , \end{aligned}$

for $\mathbf H_T = (\mathbf h_t : t=1,...,T)$ .

Ex 23. Show that for $\Pi$ the investment strategy induced by the normalized maximum likilihood policy, we have that

$\max_{\mathcal H_T} \mathcal R\! g_T ( \Pi , \mathcal P ; \mathcal H_T) \leq \max_{P\in \mathcal P} \max_{\bm H_T} \log \left( \frac{ \prod_{t=1}^T p_t(\bm h_t | \mathcal H_{t-1}) }{ \pi_T(\bm H_{T}) } \right)\, .$

Ans 23. $\begin{aligned} \exp\left\{ \max_{\mathcal H_T} \mathcal R\! g_T ( \Pi , \mathcal P ; \mathcal H_T) \right\} & \overset{Def. \ref{SI:MMR}}{=} \max_{P\in \mathcal P} \frac{ \mathcal W\! f ( P , {\mathcal H}_T) }{ \mathcal W\! f ( \Pi , {\mathcal H}_T) } \\ & \overset {[\ref{SI:6.5},\ref{SI:6}]}{=} \frac{ \sum_{\bm H_T} \prod_{t=1}^T S_t(\bm h_t) p_t(\bm h_t | \mathcal H_{t-1}) }{ \sum_{\bm H_{T+1}} \pi_T(\bm H_{T}) \prod_{\tau=1}^{t-1} S_{\tau}(\bm h_{\tau}) } \\ & \overset{[\ref{SI:4}]}{\leq} \max_{P\in \mathcal P} \max_{\bm H_T} \frac{ \prod_{t=1}^T p_t(\bm h_t | \mathcal H_{t-1}) }{ \pi_T(\bm H_{T}) }\, .\end{aligned}$

(Def. ?? = Def. 13, [??,??]= [18,22], [??]=[21].)

Ex 24. [Continued] Show that for $\mathcal I$ the set of constantly rebalanced portfolios and for $\Pi^*$ the normalized maximum likilihood policy for $\mathcal I$ , we have that

$\max_{\mathcal H_T } \mathcal R\! g_T ( \Pi^* , \mathcal I ; \mathcal H_T) \leq \mathcal V_T(\mathcal P).$

Ans 24. For a constantly rebalanced portfolio, $p_T(\mathbf H_T)=\prod_{t=1}^T p_t(\mathbf h_t)$ .

$\begin{aligned} \max_{\mathcal H_T} \mathcal R\! g_T ( \Pi^* , \mathcal I ; \mathcal H_T) & \overset{[\ref{SI:7}]}{\leq } \max_{\bm H_T} \log \left( \frac{ \max_{P\in \mathcal I} p_T(\bm H_t ) }{ \pi^*_T(\bm H_{T}) } \right)\\ & \overset{Def.\, \ref{SI:NMLF}}{=} \log \left( \sum_{\bm H'_t} \max_{P\in \mathcal P} p_T(\bm H'_t ) \right) \overset{[\ref{SI:2.5}]}{=} \mathcal V_T(\mathcal P)\, .\end{aligned}$

Here [??] = [23] , Def. ?? = Def. 4, and [??] = [6].

Ex 25. [Continued] Show that Thrm 19 holds. That is that

$\mathcal W_T(\mathcal I) = \max_{\mathcal H_T } \mathcal R\! g_T ( \Pi^* , \mathcal I ; \mathcal H_T) = \mathcal V_T(\mathcal P)\, .$

Ans 25. $\mathcal W_T(\mathcal I) := \min_{\Pi} \max_{\mathcal H_T} \mathcal R\! g_T ( \Pi , \mathcal I ; \mathcal H_T) \leq \max_{\mathcal H_T} \mathcal R\! g_T ( \Pi^* , \mathcal I ; \mathcal H_T) \overset{[\ref{SI:8}]}{\leq} \mathcal V_T(\mathcal P)\, .$ (Here [??]=[24]) and by [20] the inequalities also hold in the other direction.

Sequential Investment

Share this:

Leave a comment Cancel reply