Bounded-Velocity Stochastic Control for Dynamic Resource Allocation

The principle of optimality is a fundamental aspect of dynamic programming, which states that the optimal solution to a dynamic optimization problem can be found by combining the optimal solutions to its sub-problems. While this principle is generally applicable, it is often only taught for problems with finite or countable state spaces in order to sidestep measure-theoretic complexities. Therefore, it cannot be applied to classic models such as inventory management and dynam...

We present a methodology for obtaining explicit solutions to infinite time horizon optimal stopping problems involving general, one-dimensional, It\^o diffusions, payoff functions that need not be smooth and state-dependent discounting. This is done within a framework based on dynamic programming techniques employing variational inequalities and links to the probabilistic approaches employing $r$-excessive functions and martingale theory. The aim of this paper is to facilitat...

The most common approaches for solving stochastic resource allocation problems in the research literature is to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the impact of a decision now on the future. By contrast, common industry practice is to use a deterministic approximation of the future which is easier to understand and solve, but which is criticized for ignoring uncertainty. We show that a parameterized v...

An extended quadratic function is a quadratic function plus the indicator function of an affine set, that is, a quadratic function with embedded linear equality constraints. We show that, under some technical conditions, random convex extended quadratic functions are closed under addition, composition with an affine function, expectation, and partial minimization, that is, minimizing over some of its arguments. These properties imply that dynamic programming can be tractably ...

When sales of a product are affected by randomness in demand, retailers can use dynamic pricing strategies to maximise their profits. In this article the pricing problem is formulated as a stochastic optimal control problem, where the optimal policy can be found by solving the associated Bellman equation. The aim is to investigate Approximate Dynamic Programming algorithms for this problem. For realistic retail applications, modelling the problem and solving it to optimality ...

Recently, literature on dynamic coherent risk measures has broadened the choices for risk-sensitive performance evaluation. A running example includes Cumulative prospect theory and Conditional variance at risk. Most of them can be can be interpreted in general as a non-linear transformation of a given random variable. Non-convexity property has implied a lot of mathematical intricacies and challenges. The paper gives overview on the recent development of dynamic programming ...

In this paper, we show how a resource allocation problem can be solved through Integer Linear Programming (ILP). A detailed illustrative example is presented, together with an exhaustive overview of the mathematical model. The size of the required vectors and matrix are determined as well. The presented example can be used to learn students the fundamental basics of ILP-based resource allocation. Next, the specific benefits of the ILP approach compared to other resource alloc...

The challenge of spatial resource allocation is pervasive across various domains such as transportation, industry, and daily life. As the scale of real-world issues continues to expand and demands for real-time solutions increase, traditional algorithms face significant computational pressures, struggling to achieve optimal efficiency and real-time capabilities. In recent years, with the escalating computational power of computers, the remarkable achievements of reinforcement...

In this paper we introduce a class of Markov decision processes that arise as a natural model for many renewable resource allocation problems. Upon extending results from the inventory control literature, we prove that they admit a closed form solution and we show how to exploit this structure to speed up its computation. We consider the application of the proposed framework to several problems arising in very different domains, and as part of the ongoing effort in the emergi...

Complexity and uncertainty associated with commodity resource valuation and extraction requires stochastic control methods suitable for high dimensional states. Recent progress in duality and trajectory-wise techniques has introduced a variety of fresh ideas to this field with surprising results. This paper presents a first application of this promising development to commodity extraction problems. We introduce efficient algorithms for obtaining approximate solutions along wi...