Bounded-Velocity Stochastic Control for Dynamic Resource Allocation

We design receding horizon control strategies for stochastic discrete-time linear systems with additive (possibly) unbounded disturbances, while obeying hard bounds on the control inputs. We pose the problem of selecting an appropriate optimal controller on vector spaces of functions and show that the resulting optimization problem has a tractable convex solution. Under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the varianc...

Dynamic resource allocation problems are ubiquitous, arising in inventory management, order fulfillment, online advertising, and other applications. We initially focus on one of the simplest models of online resource allocation: the multisecretary problem. In the multisecretary problem, a decision maker sequentially hires up to $B$ out of $T$ candidates, and candidate ability values are drawn i.i.d. from a distribution $F$ on $[0,1]$. First, we investigate fundamental limits ...

In this paper we investigate possible approaches to study general time-inconsistent optimization problems without assuming the existence of optimal strategy. This leads immediately to the need to refine the concept of time-consistency as well as any method that is based on Pontryagin's Maximum Principle. The fundamental obstacle is the dilemma of having to invoke the {\it Dynamic Programming Principle} (DPP) in a time-inconsistent setting, which is contradictory in nature. Th...

This paper studies a risk-sensitive decision-making problem under uncertainty. It considers a decision-making process that unfolds over a fixed number of stages, in which a decision-maker chooses among multiple alternatives, some of which are deterministic and others are stochastic. The decision-maker's cumulative value is updated at each stage, reflecting the outcomes of the chosen alternatives. After formulating this as a stochastic control problem, we delineate the necessa...

We consider the scheduling control problem for a family of unitary networks under heavy traffic, with general interarrival and service times, probabilistic routing and infinite horizon discounted linear holding cost. A natural nonanticipativity condition for admissibility of control policies is introduced. The condition is seen to hold for a broad class of problems. Using this formulation of admissible controls and a time-transformation technique, we establish that the infimu...

When randomness in demand affects the sales of a product, retailers use dynamic pricing strategies to maximize their profits. In this article, we formulate the pricing problem as a continuous-time stochastic optimal control problem and find the optimal policy by solving the associated Hamilton-Jacobi-Bellman (HJB) equation. We propose a new approach to modelling the randomness in the dynamics of sales based on diffusion processes. The model assumes a continuum approximation t...

In this paper, we study capital allocation for dynamic risk measures, with an axiomatic approach but also by exploiting the relation between risk measures and BSDEs. Although there is a wide literature on capital allocation rules in a static setting and on dynamic risk measures, only a few recent papers on capital allocation work in a dynamic setting and, moreover, those papers mainly focus on the gradient approach. To fill this gap, we then discuss new perspectives to the ca...

This is the first in a series of papers in which we study an efficient approximation scheme for solving the Hamilton-Jacobi-Bellman equation for multi-dimensional problems in stochastic control theory. The method is a combination of a WKB style asymptotic expansion of the value function, which reduces the second order HJB partial differential equation to a hierarchy of first order PDEs, followed by a numerical algorithm to solve the first few of the resulting first order PDEs...

This article presents a new method for computing guaranteed convex and concave relaxations of nonlinear stochastic optimal control problems with final-time expected-value cost functions. This method is motivated by similar methods for deterministic optimal control problems, which have been successfully applied within spatial branch-and-bound (B&B) techniques to obtain guaranteed global optima. Relative to those methods, a key challenge here is that the expected-value cost fun...

Dynamic capacity allocation control for resource sharing networks (RSN) is studied when the networks are in heavy traffic. The goal is to minimize an ergodic cost with a linear holding cost function. Our main result shows that the optimal cost associated with an associated Brownian control problem provides a lower bound for the asymptotic ergodic cost in the RSN for any sequence of control policies. A similar result for an infinite horizon discounted cost has been previously ...