This book discusses stochastic game theory and related concepts. Topics focused upon in the book include matrix games, finite, infinite, and undiscounted stochastic games, n-player cooperative games, minimax theorem, and more. In addition to important definitions and theorems, the book provides readers with a range of problem-solving techniques and exercises. This book is of value to graduate students and readers of probability and statistics alike.

This comprehensive work examines important recent developments and modern applications in the fields of optimization, control, game theory and equilibrium programming. In particular, the concepts of equilibrium and optimality are of immense practical importance affecting decision-making problems regarding policy and strategies, and in understanding and predicting systems in different application domains, ranging from economics and engineering to military applications. The book consists of 29 survey chapters written by distinguished researchers in the above areas.

The theory of probability began in the seventeenth century with attempts to calculate the odds of winning in certain games of chance. However, it was not until the middle of the twentieth century that mathematicians de veloped general techniques for maximizing the chances of beating a casino or winning against an intelligent opponent. These methods of finding op timal strategies for a player are at the heart of the modern theories of stochastic control and stochastic games. There are numerous applications to engineering and the social sciences, but the liveliest intuition still comes from gambling. The now classic work How to Gamble If You Must: Inequalities for Stochastic Processes by Dubins and Savage (1965) uses gambling termi nology and examples to develop an elegant, deep, and quite general theory of discrete-time stochastic control. A gambler "controls" the stochastic pro cess of his or her successive fortunes by choosing which games to play and what bets to make.

This volume is based on lectures given at the NATO Advanced Study Institute on "Stochastic Games and Applications," which took place at Stony Brook, NY, USA, July 1999. It gives the editors great pleasure to present it on the occasion of L.S. Shapley's eightieth birthday, and on the fiftieth "birthday" of his seminal paper "Stochastic Games," with which this volume opens. We wish to thank NATO for the grant that made the Institute and this volume possible, and the Center for Game Theory in Economics of the State University of New York at Stony Brook for hosting this event. We also wish to thank the Hebrew University of Jerusalem, Israel, for providing continuing financial support, without which this project would never have been completed. In particular, we are grateful to our editorial assistant Mike Borns, whose work has been indispensable. We also would like to acknowledge the support of the Ecole Poly tech nique, Paris, and the Israel Science Foundation. March 2003 Abraham Neyman and Sylvain Sorin ix STOCHASTIC GAMES L.S. SHAPLEY University of California at Los Angeles Los Angeles, USA 1. Introduction In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by the two players.

Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

The theory of two-person, zero-sum differential games started at the be ginning of the 1960s with the works of R. Isaacs in the United States and L.S. Pontryagin and his school in the former Soviet Union. Isaacs based his work on the Dynamic Programming method. He analyzed many special cases of the partial differential equation now called Hamilton Jacobi-Isaacs-briefiy HJI-trying to solve them explicitly and synthe sizing optimal feedbacks from the solution. He began a study of singular surfaces that was continued mainly by J. Breakwell and P. Bernhard and led to the explicit solution of some low-dimensional but highly nontriv ial games; a recent survey of this theory can be found in the book by J. Lewin entitled Differential Games (Springer, 1994). Since the early stages of the theory, several authors worked on making the notion of value of a differential game precise and providing a rigorous derivation of the HJI equation, which does not have a classical solution in most cases; we mention here the works of W. Fleming, A. Friedman (see his book, Differential Games, Wiley, 1971), P.P. Varaiya, E. Roxin, R.J. Elliott and N.J. Kalton, N.N. Krasovskii, and A.I. Subbotin (see their book Po sitional Differential Games, Nauka, 1974, and Springer, 1988), and L.D. Berkovitz. A major breakthrough was the introduction in the 1980s of two new notions of generalized solution for Hamilton-Jacobi equations, namely, viscosity solutions, by M.G. Crandall and P.-L.

This book gathers the most essential results, including recent ones, on linear-quadratic optimal control problems, which represent an important aspect of stochastic control. It presents results for two-player differential games and mean-field optimal control problems in the context of finite and infinite horizon problems, and discusses a number of new and interesting issues. Further, the book identifies, for the first time, the interconnections between the existence of open-loop and closed-loop Nash equilibria, solvability of the optimality system, and solvability of the associated Riccati equation, and also explores the open-loop solvability of mean-filed linear-quadratic optimal control problems. Although the content is largely self-contained, readers should have a basic grasp of linear algebra, functional analysis and stochastic ordinary differential equations. The book is mainly intended for senior undergraduate and graduate students majoring in applied mathematics who are interested in stochastic control theory. However, it will also appeal to researchers in other related areas, such as engineering, management, finance/economics and the social sciences.