Stochastic processes are mathematical models of random phenomena that evolve according to prescribed dynamics. Processes commonly used in applications are Markov chains in discrete and continuous time, renewal and regenerative processes, Poisson processes, and Brownian motion. This volume gives an in-depth description of the structure and basic properties of these stochastic processes. A main focus is on equilibrium distributions, strong laws of large numbers, and ordinary and functional central limit theorems for cost and performance parameters. Although these results differ for various processes, they have a common trait of being limit theorems for processes with regenerative increments. Extensive examples and exercises show how to formulate stochastic models of systems as functions of a system’s data and dynamics, and how to represent and analyze cost and performance measures. Topics include stochastic networks, spatial and space-time Poisson processes, queueing, reversible processes, simulation, Brownian approximations, and varied Markovian models. The technical level of the volume is between that of introductory texts that focus on highlights of applied stochastic processes, and advanced texts that focus on theoretical aspects of processes.

Stochastic processes are mathematical models of random phenomena that evolve according to prescribed dynamics. Processes commonly used in applications are Markov chains in discrete and continuous time, renewal and regenerative processes, Poisson processes, and Brownian motion. This volume gives an in-depth description of the structure and basic properties of these stochastic processes. A main focus is on equilibrium distributions, strong laws of large numbers, and ordinary and functional central limit theorems for cost and performance parameters. Although these results differ for various processes, they have a common trait of being limit theorems for processes with regenerative increments. Extensive examples and exercises show how to formulate stochastic models of systems as functions of a system’s data and dynamics, and how to represent and analyze cost and performance measures. Topics include stochastic networks, spatial and space-time Poisson processes, queueing, reversible processes, simulation, Brownian approximations, and varied Markovian models. The technical level of the volume is between that of introductory texts that focus on highlights of applied stochastic processes, and advanced texts that focus on theoretical aspects of processes.

This book uses a distinctly applied framework to present the most important topics in stochastic processes, including Gaussian and Markovian processes, Markov Chains, Poisson processes, Brownian motion and queueing theory. The book also examines in detail special diffusion processes, with implications for finance, various generalizations of Poisson processes, and renewal processes. It contains numerous examples and approximately 350 advanced problems that reinforce both concepts and applications. Entertaining mini-biographies of mathematicians give an enriching historical context. The book includes statistical tables and solutions to the even-numbered problems at the end.

This is a textbook for advanced undergraduate students and beginning graduate students in applied mathematics. It presents the basic mathematical foundations of stochastic analysis (probability theory and stochastic processes) as well as some important practical tools and applications (e.g., the connection with differential equations, numerical methods, path integrals, random fields, statistical physics, chemical kinetics, and rare events). The book strikes a nice balance between mathematical formalism and intuitive arguments, a style that is most suited for applied mathematicians. Readers can learn both the rigorous treatment of stochastic analysis as well as practical applications in modeling and simulation. Numerous exercises nicely supplement the main exposition.

The third edition of this volume improves on the last edition by condensing the material and organizing it into a more teachable format. It provides more in-depth coverage of Markov chains and simple Markov process and gives added emphasis to statistical inference in stochastic processes.

Building upon the previous editions, this textbook is a first course in stochastic processes taken by undergraduate and graduate students (MS and PhD students from math, statistics, economics, computer science, engineering, and finance departments) who have had a course in probability theory. It covers Markov chains in discrete and continuous time, Poisson processes, renewal processes, martingales, and option pricing. One can only learn a subject by seeing it in action, so there are a large number of examples and more than 300 carefully chosen exercises to deepen the reader’s understanding. Drawing from teaching experience and student feedback, there are many new examples and problems with solutions that use TI-83 to eliminate the tedious details of solving linear equations by hand, and the collection of exercises is much improved, with many more biological examples. Originally included in previous editions, material too advanced for this first course in stochastic processes has been eliminated while treatment of other topics useful for applications has been expanded. In addition, the ordering of topics has been improved; for example, the difficult subject of martingales is delayed until its usefulness can be applied in the treatment of mathematical finance.

Applied Stochastic Processes presents a concise, graduate-level treatment of the subject, emphasizing applications and practical computation. It also establishes the complete mathematical theory in an accessible way. After reviewing basic probability, the text covers Poisson processes, renewal processes, discrete- and continuous-time Markov chains, and Brownian motion. It also offers an introduction to stochastic differential equations. While the main applications described are queues, the book also considers other examples, such as the mathematical model of a single stock market. With exercises in most sections, this book provides a clear, practical introduction for beginning graduate students. The material is presented in a straightforward manner using short, motivating examples. In addition, the author develops the mathematical theory with a strong emphasis on probability intuition.

This self-contained, practical, entry-level text integrates the basic principles of applied mathematics, applied probability, and computational science for a clear presentation of stochastic processes and control for jump diffusions in continuous time. The author covers the important problem of controlling these systems and, through the use of a jump calculus construction, discusses the strong role of discontinuous and nonsmooth properties versus random properties in stochastic systems.

This book presents various results and techniques from the theory of stochastic processes that are useful in the study of stochastic problems in the natural sciences. The main focus is analytical methods, although numerical methods and statistical inference methodologies for studying diffusion processes are also presented. The goal is the development of techniques that are applicable to a wide variety of stochastic models that appear in physics, chemistry and other natural sciences. Applications such as stochastic resonance, Brownian motion in periodic potentials and Brownian motors are studied and the connection between diffusion processes and time-dependent statistical mechanics is elucidated. The book contains a large number of illustrations, examples, and exercises. It will be useful for graduate-level courses on stochastic processes for students in applied mathematics, physics and engineering. Many of the topics covered in this book (reversible diffusions, convergence to equilibrium for diffusion processes, inference methods for stochastic differential equations, derivation of the generalized Langevin equation, exit time problems) cannot be easily found in textbook form and will be useful to both researchers and students interested in the applications of stochastic processes.