Optim example: HJB equation TeX

pricing.tex

\documentclass{article}
\usepackage{amssymb,amsmath}
\usepackage{filecontents}
\usepackage{url}

\DeclareMathOperator*{\argmax}{arg\,max}
\author{Asbj{\o}rn Nilsen Riseth}
\date{\today}
\title{Optim example: Solve HJB equation}

\begin{document}

\begin{filecontents}{\jobname.bib}
@article{debrabant2013semi,
  title={Semi-Lagrangian schemes for linear and fully non-linear diffusion equations},
  author={Debrabant, Kristian and Jakobsen, Espen},
  journal={Mathematics of Computation},
  volume={82},
  number={283},
  pages={1433--1462},
  year={2013}
}

@book{pham2009continuous,
  title={Continuous-time stochastic control and optimization with financial applications},
  author={Pham, Huy{\^e}n},
  volume={61},
  year={2009},
  publisher={Springer Science \& Business Media}
}
\end{filecontents}

\maketitle

\section{Problem overview}
This document explains the mathematical problem solved by the Optim
example at
\url{https://gist.github.com/anriseth/d27f51d2e72a937f5752bb9fc0c0fdbb}.

The example file finds the function the
function $v:[0,T]\times \mathbb R_{\geq 0}\to \mathbb R$ that solves
the Hamilton-Jacobi-Bellman equation given by
\begin{align}
  v_t+\sup_{a\in A}\left\{\frac{\gamma^2}{2}{q(a)}^2v_{xx}-q(a)v_x +
  aq(a)\right\} &= 0,\\
  v(T,x) &=-Cx,\\
  v(t,0)&=0.
\end{align}
In the file, $A=[0,1]$, $q(a)=1-a$, $C=1$, and $\gamma=5\times 10^{-2}$.
The PDE is discretised and solved backwards in time using a Semi-Lagrangian scheme for
diffusion problems, as described in~\cite{debrabant2013semi}.
We use Optim to solve the optimisation problem in this nonlinear PDE,
and conceptually what we do at a given time $t$, is to find
$v(t-\Delta t, x)$ from the optimisation
\begin{equation}
  v(t-\Delta t,x)
  = v(t, x) + \Delta t \sup_{a\in A}\left\{\frac{\gamma^2}{2}{q(a)}^2v_{xx}-q(a)v_x +
    aq(a)\right\}.
\end{equation}
Thus, for each time step and each point on the spatial grid, we must
solve a constrained optimisation problem. Optim works very well for
this purpose.

The file in the URL above approximates $v(t,x)$ and prints that, as
well as the control function $a(t,x)$, given by
\begin{equation}\label{eq:afun}
  a(t,x)=\argmax_{a\in A}\left\{\frac{\gamma^2}{2}{q(a)}^2v_{xx}-q(a)v_x +
    aq(a)\right\}.
\end{equation}

\subsection{Problem motivation}
The PDE above arises from a stochastic optimal control problem.
We wish to dynamically set the price of a product, in order to
maximise the expected profit accrued from the revenue over a period $[0,T]$ minus
the cost of handling unsold stock at time $T$.
Let $X^\alpha(t)$ denote the amount of stock at time $t$, following a
pricing strategy $\alpha(t)$. Given an expected demand function
$q(a)$, and a Brownian motion $W(t)$, the stock evolves according to
the SDE
\begin{equation}\label{eq:sde_x}
  dX^\alpha(t)=-q(\alpha(t))(dt+\gamma dW(t)),\qquad\text{when } X^\alpha(t)>0.
\end{equation}
We wish to maximise profit by finding pricing strategies that decides
the price at time $t$, given the amount of stock left at this time,
$X^\alpha(t)$. Call $\mathcal A$ the set of processes
of the form $\alpha(t)=a(t,X^\alpha(t))$ for measurable functions
$a(t,x)$ taking values in $A$.
Let $C>0$ denote the cost per unit unsold stock, and let $T^s\leq T$
be the hitting time for $X^\alpha(t)=0$.
The stochastic optimal control problem is thus to find an $\alpha(t)$
that solves
\begin{equation}
  \max_{\alpha\in \mathcal A}\mathbb E\left[
    \int_0^{T^s} \alpha(t)q(\alpha(t))dt-CX^\alpha(T)
  \right].
\end{equation}
This is an infinite-dimensional optimisation problem, but it turns out
that one can find $\alpha$ from the function $a(t,x)$
in~\eqref{eq:afun}. For a reference, see~\cite{pham2009continuous}.

\textbf{Final note.}
The model for the uncertainty in the evolution of stock
in~\eqref{eq:sde_x} does not really make sense, as it allows negative
sales with probability close to $0.5$ over small timesteps.


\bibliographystyle{plain}
\bibliography{\jobname}

\end{document}