During the past decade there has been an explosion in computation and information technology. With it have come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics. Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It should be a valuable resource for statisticians and anyone interested in data mining in science or industry. The book’s coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book. This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for “wide” data (p bigger than n), including multiple testing and false discovery rates. Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie co-developed much of the statistical modeling software and environment in R/S-PLUS and invented principal curves and surfaces. Tibshirani proposed the lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, projection pursuit and gradient boosting.

This classic textbook builds theoretical statistics from the first principles of probability theory. Starting from the basics of probability, the authors develop the theory of statistical inference using techniques, definitions, and concepts that are statistical and natural extensions, and consequences, of previous concepts. It covers all topics from a standard inference course including: distributions, random variables, data reduction, point estimation, hypothesis testing, and interval estimation. Features The classic graduate-level textbook on statistical inference Develops elements of statistical theory from first principles of probability Written in a lucid style accessible to anyone with some background in calculus Covers all key topics of a standard course in inference Hundreds of examples throughout to aid understanding Each chapter includes an extensive set of graduated exercises Statistical Inference, Second Edition is primarily aimed at graduate students of statistics, but can be used by advanced undergraduate students majoring in statistics who have a solid mathematics background. It also stresses the more practical uses of statistical theory, being more concerned with understanding basic statistical concepts and deriving reasonable statistical procedures, while less focused on formal optimality considerations. This is a reprint of the second edition originally published by Cengage Learning, Inc. in 2001.

The twenty-first century has seen a breathtaking expansion of statistical methodology, both in scope and influence. 'Data science' and 'machine learning' have become familiar terms in the news, as statistical methods are brought to bear upon the enormous data sets of modern science and commerce. How did we get here? And where are we going? How does it all fit together? Now in paperback and fortified with exercises, this book delivers a concentrated course in modern statistical thinking. Beginning with classical inferential theories - Bayesian, frequentist, Fisherian - individual chapters take up a series of influential topics: survival analysis, logistic regression, empirical Bayes, the jackknife and bootstrap, random forests, neural networks, Markov Chain Monte Carlo, inference after model selection, and dozens more. The distinctly modern approach integrates methodology and algorithms with statistical inference. Each chapter ends with class-tested exercises, and the book concludes with speculation on the future direction of statistics and data science.

The book is a collection of 80 short and self-contained lectures covering most of the topics that are usually taught in intermediate courses in probability theory and mathematical statistics. There are hundreds of examples, solved exercises and detailed derivations of important results. The step-by-step approach makes the book easy to understand and ideal for self-study. One of the main aims of the book is to be a time saver: it contains several results and proofs, especially on probability distributions, that are hard to find in standard references and are scattered here and there in more specialistic books. The topics covered by the book are as follows. PART 1 - MATHEMATICAL TOOLS: set theory, permutations, combinations, partitions, sequences and limits, review of differentiation and integration rules, the Gamma and Beta functions. PART 2 - FUNDAMENTALS OF PROBABILITY: events, probability, independence, conditional probability, Bayes' rule, random variables and random vectors, expected value, variance, covariance, correlation, covariance matrix, conditional distributions and conditional expectation, independent variables, indicator functions. PART 3 - ADDITIONAL TOPICS IN PROBABILITY THEORY: probabilistic inequalities, construction of probability distributions, transformations of probability distributions, moments and cross-moments, moment generating functions, characteristic functions. PART 4 - PROBABILITY DISTRIBUTIONS: Bernoulli, binomial, Poisson, uniform, exponential, normal, Chi-square, Gamma, Student's t, F, multinomial, multivariate normal, multivariate Student's t, Wishart. PART 5 - MORE DETAILS ABOUT THE NORMAL DISTRIBUTION: linear combinations, quadratic forms, partitions. PART 6 - ASYMPTOTIC THEORY: sequences of random vectors and random variables, pointwise convergence, almost sure convergence, convergence in probability, mean-square convergence, convergence in distribution, relations between modes of convergence, Laws of Large Numbers, Central Limit Theorems, Continuous Mapping Theorem, Slutsky's Theorem. PART 7 - FUNDAMENTALS OF STATISTICS: statistical inference, point estimation, set estimation, hypothesis testing, statistical inferences about the mean, statistical inferences about the variance.

Aimed at advanced undergraduates and graduate students in mathematics and related disciplines, this engaging textbook gives a concise account of the main approaches to inference, with particular emphasis on the contrasts between them. It is the first textbook to synthesize contemporary material on computational topics with basic mathematical theory.

Taken literally, the title "All of Statistics" is an exaggeration. But in spirit, the title is apt, as the book does cover a much broader range of topics than a typical introductory book on mathematical statistics. This book is for people who want to learn probability and statistics quickly. It is suitable for graduate or advanced undergraduate students in computer science, mathematics, statistics, and related disciplines. The book includes modern topics like non-parametric curve estimation, bootstrapping, and classification, topics that are usually relegated to follow-up courses. The reader is presumed to know calculus and a little linear algebra. No previous knowledge of probability and statistics is required. Statistics, data mining, and machine learning are all concerned with collecting and analysing data.

An Introduction to Statistical Learning provides an accessible overview of the field of statistical learning, an essential toolset for making sense of the vast and complex data sets that have emerged in fields ranging from biology to finance, marketing, and astrophysics in the past twenty years. This book presents some of the most important modeling and prediction techniques, along with relevant applications. Topics include linear regression, classification, resampling methods, shrinkage approaches, tree-based methods, support vector machines, clustering, deep learning, survival analysis, multiple testing, and more. Color graphics and real-world examples are used to illustrate the methods presented. This book is targeted at statisticians and non-statisticians alike, who wish to use cutting-edge statistical learning techniques to analyze their data. Four of the authors co-wrote An Introduction to Statistical Learning, With Applications in R (ISLR), which has become a mainstay of undergraduate and graduate classrooms worldwide, as well as an important reference book for data scientists. One of the keys to its success was that each chapter contains a tutorial on implementing the analyses and methods presented in the R scientific computing environment. However, in recent years Python has become a popular language for data science, and there has been increasing demand for a Python-based alternative to ISLR. Hence, this book (ISLP) covers the same materials as ISLR but with labs implemented in Python. These labs will be useful both for Python novices, as well as experienced users.

A concise and self-contained introduction to causal inference, increasingly important in data science and machine learning. The mathematization of causality is a relatively recent development, and has become increasingly important in data science and machine learning. This book offers a self-contained and concise introduction to causal models and how to learn them from data. After explaining the need for causal models and discussing some of the principles underlying causal inference, the book teaches readers how to use causal models: how to compute intervention distributions, how to infer causal models from observational and interventional data, and how causal ideas could be exploited for classical machine learning problems. All of these topics are discussed first in terms of two variables and then in the more general multivariate case. The bivariate case turns out to be a particularly hard problem for causal learning because there are no conditional independences as used by classical methods for solving multivariate cases. The authors consider analyzing statistical asymmetries between cause and effect to be highly instructive, and they report on their decade of intensive research into this problem. The book is accessible to readers with a background in machine learning or statistics, and can be used in graduate courses or as a reference for researchers. The text includes code snippets that can be copied and pasted, exercises, and an appendix with a summary of the most important technical concepts.