This book for beginning graduate students presents a course on stochastic games and the mathematical methods used in their analysis.

This book discusses stochastic game theory and related concepts. Topics focused upon in the book include matrix games, finite, infinite, and undiscounted stochastic games, n-player cooperative games, minimax theorem, and more. In addition to important definitions and theorems, the book provides readers with a range of problem-solving techniques and exercises. This book is of value to graduate students and readers of probability and statistics alike.

The theory of probability began in the seventeenth century with attempts to calculate the odds of winning in certain games of chance. However, it was not until the middle of the twentieth century that mathematicians de veloped general techniques for maximizing the chances of beating a casino or winning against an intelligent opponent. These methods of finding op timal strategies for a player are at the heart of the modern theories of stochastic control and stochastic games. There are numerous applications to engineering and the social sciences, but the liveliest intuition still comes from gambling. The now classic work How to Gamble If You Must: Inequalities for Stochastic Processes by Dubins and Savage (1965) uses gambling termi nology and examples to develop an elegant, deep, and quite general theory of discrete-time stochastic control. A gambler "controls" the stochastic pro cess of his or her successive fortunes by choosing which games to play and what bets to make.

This volume is based on lectures given at the NATO Advanced Study Institute on "Stochastic Games and Applications," which took place at Stony Brook, NY, USA, July 1999. It gives the editors great pleasure to present it on the occasion of L.S. Shapley's eightieth birthday, and on the fiftieth "birthday" of his seminal paper "Stochastic Games," with which this volume opens. We wish to thank NATO for the grant that made the Institute and this volume possible, and the Center for Game Theory in Economics of the State University of New York at Stony Brook for hosting this event. We also wish to thank the Hebrew University of Jerusalem, Israel, for providing continuing financial support, without which this project would never have been completed. In particular, we are grateful to our editorial assistant Mike Borns, whose work has been indispensable. We also would like to acknowledge the support of the Ecole Poly tech nique, Paris, and the Israel Science Foundation. March 2003 Abraham Neyman and Sylvain Sorin ix STOCHASTIC GAMES L.S. SHAPLEY University of California at Los Angeles Los Angeles, USA 1. Introduction In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by the two players.

To honor Rafail Z. Khasminskii, on his seventy-fifth birthday, for his contributions to stochastic processes and nonparametric estimation theory an IMA participating institution conference entitled "Conference on Asymptotic Analysis in Stochastic Processes, Nonparametric Estimation, and Related Problems" was held. This volume commemorates this special event. Dedicated to Professor Khasminskii, it consists of nine papers on various topics in probability and statistics.

Stochastic games have an element of chance: the state of the next round is determined probabilistically depending upon players' actions and the current state. Successful players need to balance the need for short-term payoffs while ensuring future opportunities remain high. The various techniques needed to analyze these often highly non-trivial games are a showcase of attractive mathematics, including methods from probability, differential equations, algebra, and combinatorics. This book presents a course on the theory of stochastic games going from the basics through to topics of modern research, focusing on conceptual clarity over complete generality. Each of its chapters introduces a new mathematical tool – including contracting mappings, semi-algebraic sets, infinite orbits, and Ramsey's theorem, among others – before discussing the game-theoretic results they can be used to obtain. The author assumes no more than a basic undergraduate curriculum and illustrates the theory with numerous examples and exercises, with solutions available online.

Game theory involves multi-person decision making and differential dynamic game theory has been widely applied to n-person decision making problems, which are stimulated by a vast number of applications. This book addresses the gap to discuss general stochastic n-person noncooperative and cooperative game theory with wide applications to control systems, signal processing systems, communication systems, managements, financial systems, and biological systems. H∞ game strategy, n-person cooperative and noncooperative game strategy are discussed for linear and nonlinear stochastic systems along with some computational algorithms developed to efficiently solve these game strategies.