Bounded-Velocity Stochastic Control for Dynamic Resource Allocation

We develop methods to solve general optimal stopping problems with opportunities to stop that arrive randomly. Such problems occur naturally in applications with market frictions. Pivotal to our approach is that our methods operate on random rather than deterministic time scales. This enables us to convert the original problem into an equivalent discrete-time optimal stopping problem with $\mathbb{N}_{0}$-valued stopping times and a possibly infinite horizon. To numerically s...

This paper presents a class of Dynamic Multi-Armed Bandit problems where the reward can be modeled as the noisy output of a time varying linear stochastic dynamic system that satisfies some boundedness constraints. The class allows many seemingly different problems with time varying option characteristics to be considered in a single framework. It also opens up the possibility of considering many new problems of practical importance. For instance it affords the simultaneous c...

For years, there has been interest in approximation methods for solving dynamic programming problems, because of the inherent complexity in computing optimal solutions characterized by Bellman's principle of optimality. A wide range of approximate dynamic programming (ADP) methods now exists. It is of great interest to guarantee that the performance of an ADP scheme be at least some known fraction, say $\beta$, of optimal. This paper introduces a general approach to bounding ...

In this paper, we investigate dynamic optimization problems featuring both stochastic control and optimal stopping in a finite time horizon. The paper aims to develop new methodologies, which are significantly different from those of mixed dynamic optimal control and stopping problems in the existing literature, to study a manager's decision. We formulate our model to a free boundary problem of a fully nonlinear equation. Furthermore, by means of a dual transformation for the...

We consider an investor facing a classical portfolio problem of optimal investment in a log-Brownian stock and a fixed-interest bond, but constrained to choose portfolio and consumption strategies that reduce a dynamic shortfall risk measure. For continuous- and discrete-time financial markets we investigate the loss in expected utility of intermediate consumption and terminal wealth caused by imposing a dynamic risk constraint. We derive the dynamic programming equations for...

In this paper we consider multiple constrained resource allocation problems, where the constraints can be specified by formulating activity dependency restrictions or by using game-theoretic models. All the problems are focused on generic resources, with a few exceptions which consider financial resources in particular. The problems consider low-risk circumstances and the values of the uncertain variables which are used by the algorithms are the expected values of the variabl...

In this paper, we consider the problem of optimization of a portfolio consisting of securities. An investor with an initial capital, is interested in constructing a portfolio of securities. If the prices of securities change, the investor shall decide on reallocation of the portfolio. At each moment of time, the prices of securities change and the investor is interested in constructing a dynamic portfolio of securities. The investor wishes to maximize the value of his portfol...

This paper studies the problem of optimally extracting nonrenewable natural resource in light of various financial and economic restrictions and constraints. Taking into account the fact that the market values of the main natural resources i.e. oil, natural gas, copper,...,etc, fluctuate randomly following global and seasonal macroeconomic parameters, these values are modeled using Markov switching L\'evy processes. We formulate this problem as finite-time horizon combined op...

Optimization underpins many of the challenges that science and technology face on a daily basis. Recent years have witnessed a major shift from traditional optimization paradigms grounded on batch algorithms for medium-scale problems to challenging dynamic, time-varying, and even huge-size settings. This is driven by technological transformations that converted infrastructural and social platforms into complex and dynamic networked systems with even pervasive sensing and comp...

Optimal resource allocation is gaining a renewed interest due its relevance as a core problem in managing, over time, cloud and high-performance computing facilities. Semi-Bandit Feedback (SBF) is the reference method for efficiently solving this problem. In this paper we propose (i) an extension of the optimal resource allocation to a more general class of problems, specifically with resources availability changing over time, and (ii) Bayesian Optimization as a more efficien...