This book presents the latest findings on stochastic dynamic programming models and on solving optimal control problems in networks. It includes the authors’ new findings on determining the optimal solution of discrete optimal control problems in networks and on solving game variants of Markov decision problems in the context of computational networks. First, the book studies the finite state space of Markov processes and reviews the existing methods and algorithms for determining the main characteristics in Markov chains, before proposing new approaches based on dynamic programming and combinatorial methods. Chapter two is dedicated to infinite horizon stochastic discrete optimal control models and Markov decision problems with average and expected total discounted optimization criteria, while Chapter three develops a special game-theoretical approach to Markov decision processes and stochastic discrete optimal control problems. In closing, the book’s final chapter is devoted to finite horizon stochastic control problems and Markov decision processes. The algorithms developed represent a valuable contribution to the important field of computational network theory.

Stochastic discrete-event systems (SDES) capture the randomness in choices due to activity delays and the probabilities of decisions. This book delivers a comprehensive overview on modeling with a quantitative evaluation of SDES. It presents an abstract model class for SDES as a pivotal unifying result and details important model classes. The book also includes nontrivial examples to explain real-world applications of SDES.

A recent development in SDC-related problems is the establishment of intelligent SDC models and the intensive use of LMI-based convex optimization methods. Within this theoretical framework, control parameter determination can be designed and stability and robustness of closed-loop systems can be analyzed. This book describes the new framework of SDC system design and provides a comprehensive description of the modelling of controller design tools and their real-time implementation. It starts with a review of current research on SDC and moves on to some basic techniques for modelling and controller design of SDC systems. This is followed by a description of controller design for fixed-control-structure SDC systems, PDF control for general input- and output-represented systems, filtering designs, and fault detection and diagnosis (FDD) for SDC systems. Many new LMI techniques being developed for SDC systems are shown to have independent theoretical significance for robust control and FDD problems.

Complex Social Networks is a newly emerging (hot) topic with applications in a variety of domains, such as communication networks, engineering networks, social networks, and biological networks. In the last decade, there has been an explosive growth of research on complex real-world networks, a theme that is becoming pervasive in many disciplines, ranging from mathematics and computer science to the social and biological sciences. Optimization of complex communication networks requires a deep understanding of the interplay between the dynamics of the physical network and the information dynamics within the network. Although there are a few books addressing social networks or complex networks, none of them has specially focused on the optimization perspective of studying these networks. This book provides the basic theory of complex networks with several new mathematical approaches and optimization techniques to design and analyze dynamic complex networks. A wide range of applications and optimization problems derived from research areas such as cellular and molecular chemistry, operations research, brain physiology, epidemiology, and ecology.

Opening new directions in research in both discrete event dynamic systems as well as in stochastic control, this volume focuses on a wide class of control and of optimization problems over sequences of integer numbers. This is a counterpart of convex optimization in the setting of discrete optimization. The theory developed is applied to the control of stochastic discrete-event dynamic systems. Some applications are admission, routing, service allocation and vacation control in queuing networks. Pure and applied mathematicians will enjoy reading the book since it brings together many disciplines in mathematics: combinatorics, stochastic processes, stochastic control and optimization, discrete event dynamic systems, algebra.

From foundations to state-of-the-art; the tools and philosophy you need to build network models.

This book constitutes the conference proceedings of the 5th International Conference on Algorithmic Decision Theory , ADT 2017, held in Luxembourg, in October 2017.The 22 full papers presented together with 6 short papers, 4 keynote abstracts, and 6 Doctoral Consortium papers, were carefully selected from 45 submissions. The papers are organized in topical sections on preferences and multi-criteria decision aiding; decision making and voting; game theory and decision theory; and allocation and matching.

REINFORCEMENT LEARNING AND STOCHASTIC OPTIMIZATION Clearing the jungle of stochastic optimization Sequential decision problems, which consist of “decision, information, decision, information,” are ubiquitous, spanning virtually every human activity ranging from business applications, health (personal and public health, and medical decision making), energy, the sciences, all fields of engineering, finance, and e-commerce. The diversity of applications attracted the attention of at least 15 distinct fields of research, using eight distinct notational systems which produced a vast array of analytical tools. A byproduct is that powerful tools developed in one community may be unknown to other communities. Reinforcement Learning and Stochastic Optimization offers a single canonical framework that can model any sequential decision problem using five core components: state variables, decision variables, exogenous information variables, transition function, and objective function. This book highlights twelve types of uncertainty that might enter any model and pulls together the diverse set of methods for making decisions, known as policies, into four fundamental classes that span every method suggested in the academic literature or used in practice. Reinforcement Learning and Stochastic Optimization is the first book to provide a balanced treatment of the different methods for modeling and solving sequential decision problems, following the style used by most books on machine learning, optimization, and simulation. The presentation is designed for readers with a course in probability and statistics, and an interest in modeling and applications. Linear programming is occasionally used for specific problem classes. The book is designed for readers who are new to the field, as well as those with some background in optimization under uncertainty. Throughout this book, readers will find references to over 100 different applications, spanning pure learning problems, dynamic resource allocation problems, general state-dependent problems, and hybrid learning/resource allocation problems such as those that arose in the COVID pandemic. There are 370 exercises, organized into seven groups, ranging from review questions, modeling, computation, problem solving, theory, programming exercises and a “diary problem” that a reader chooses at the beginning of the book, and which is used as a basis for questions throughout the rest of the book.

This book addresses issues associated with the interface of computing, optimisation, econometrics and financial modeling, emphasizing computational optimisation methods and techniques. The first part addresses optimisation problems and decision modeling, plus applications of supply chain and worst-case modeling and advances in methodological aspects of optimisation techniques. The second part covers optimisation heuristics, filtering, signal extraction and time series models. The final part discusses optimisation in portfolio selection and real option modeling.

This book offers a comprehensive review of smart technologies and perspectives on their application in urban engineering. It covers a wide range of applications, from transport and energy management to digital manufacturing, smart city, environment, and sustainable development, providing readers with new ideas for future research and collaborations. This book presents select papers from the International Conference on Smart Technologies in Urban Engineering (STUE-2022), held to commemorate the 100th anniversary of the O.M. Beketov National University of Urban Economy in Kharkiv, Ukraine, on June 9–11, 2022. All the contributions offer plenty of valuable information and would be of great benefit to the experience exchange among scientists in urban engineering.