Entropy, mutual information and divergence measure the randomness, dependence and dissimilarity, respectively, of random objects. In addition to their prominent role in information theory, they have found numerous applications, among others, in probability theory statistics, physics, chemistry, molecular biology, ecology, bioinformatics, neuroscience, machine learning, linguistics, and finance. Many of these applications require a universal estimate of information measures which does not assume knowledge of the statistical properties of the observed data. Over the past few decades, several nonparametric algorithms have been proposed to estimate information measures. Universal Estimation of Information Measures for Analog Sources presents a comprehensive survey of universal estimation of information measures for memoryless analog (real-valued or real vector-valued) sources with an emphasis on the estimation of mutual information and divergence and their applications. The book reviews the consistency of the universal algorithms and the corresponding sufficient conditions as well as their speed of convergence. Universal Estimation of Information Measures for Analog Sources provides a comprehensive review of an increasingly important topic in Information Theory. It will be of interest to students, practitioners and researchers working in Information Theory

Entropies and entropy-like quantities play an increasing role in modern non-linear data analysis. Fields that benefit from this application range from biosignal analysis to econophysics and engineering. This issue is a collection of papers touching on different aspects of entropy measures in data analysis, as well as theoretical and computational analyses. The relevant topics include the difficulty to achieve adequate application of entropy measures and the acceptable parameter choices for those entropy measures, entropy-based coupling, and similarity analysis, along with the utilization of entropy measures as features in automatic learning and classification. Various real data applications are given.

Analysis of information transfer has found rapid adoption in neuroscience, where a highly dynamic transfer of information continuously runs on top of the brain's slowly-changing anatomical connectivity. Measuring such transfer is crucial to understanding how flexible information routing and processing give rise to higher cognitive function. Directed Information Measures in Neuroscience reviews recent developments of concepts and tools for measuring information transfer, their application to neurophysiological recordings and analysis of interactions. Written by the most active researchers in the field the book discusses the state of the art, future prospects and challenges on the way to an efficient assessment of neuronal information transfer. Highlights include the theoretical quantification and practical estimation of information transfer, description of transfer locally in space and time, multivariate directed measures, information decomposition among a set of stimulus/responses variables and the relation between interventional and observational causality. Applications to neural data sets and pointers to open source software highlight the usefulness of these measures in experimental neuroscience. With state-of-the-art mathematical developments, computational techniques and applications to real data sets, this book will be of benefit to all graduate students and researchers interested in detecting and understanding the information transfer between components of complex systems.

From the reviews: "Bioinformaticians are facing the challenge of how to handle immense amounts of raw data, [...] and render them accessible to scientists working on a wide variety of problems. [This book] can be such a tool." IEEE Engineering in Medicine and Biology

Information Theory and Statistics: A Tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting. The topics covered include large deviations, hypothesis testing, maximum likelihood estimation in exponential families, analysis of contingency tables, and iterative algorithms with an "information geometry" background. Also, an introduction is provided to the theory of universal coding, and to statistical inference via the minimum description length principle motivated by that theory. The tutorial does not assume the reader has an in-depth knowledge of Information Theory or statistics. As such, Information Theory and Statistics: A Tutorial, is an excellent introductory text to this highly-important topic in mathematics, computer science and electrical engineering. It provides both students and researchers with an invaluable resource to quickly get up to speed in the field.

Measurement plays a fundamental role both in physical and behavioral sciences, as well as in engineering and technology: it is the link between abstract models and empirical reality and is a privileged method of gathering information from the real world. Is it possible to develop a single theory of measurement for the various domains of science and technology in which measurement is involved? This book takes the challenge by addressing the following main issues: What is the meaning of measurement? How do we measure? What can be measured? A theoretical framework that could truly be shared by scientists in different fields, ranging from physics and engineering to psychology is developed. The future in fact will require greater collaboration between science and technology and between different sciences. Measurement, which played a key role in the birth of modern science, can act as an essential interdisciplinary tool and language for this new scenario. A sound theoretical basis for addressing key problems in measurement is provided. These include perceptual measurement, the evaluation of uncertainty, the evaluation of inter-comparisons, the analysis of risks in decision-making and the characterization of dynamical measurement. Currently, increasing attention is paid to these issues due to their scientific, technical, economic and social impact. The book proposes a unified probabilistic approach to them which may allow more rational and effective solutions to be reached. Great care was taken to make the text as accessible as possible in several ways. Firstly, by giving preference to as interdisciplinary a terminology as possible; secondly, by carefully defining and discussing all key terms. This ensures that a wide readership, including people from different mathematical backgrounds and different understandings of measurement can all benefit from this work. Concerning mathematics, all the main results are preceded by intuitive discussions and illustrated by simple examples. Moreover, precise proofs are always included in order to enable the more demanding readers to make conscious and creative use of these ideas, and also to develop new ones. The book demonstrates that measurement, which is commonly understood to be a merely experimental matter, poses theoretical questions which are no less challenging than those arising in other, apparently more theoretical, disciplines.

This book discusses the theories, methods, and application techniques of the measurement data mathematical modeling and parameter estimation. It seeks to build a bridge between mathematical theory and engineering practice in the measurement data processing field so theoretical researchers and technical engineers can communicate. It is organized with abundant materials, such as illustrations, tables, examples, and exercises. The authors create examples to apply mathematical theory innovatively to measurement and control engineering. Not only does this reference provide theoretical knowledge, it provides information on first hand experiences.

"This book is highly recommended for all those whose interests lie in the fields that deal with any kind of information measures. It will also find readers in the field of functional analysis..".Mathematical Reviews

This book is devoted to current problems of artificial and computational intelligence including decision-making systems. Collecting, analysis, and processing information are the current directions of modern computer science. Development of new modern information and computer technologies for data analysis and processing in various fields of data mining and machine learning creates the conditions for increasing effectiveness of the information processing by both the decrease of time and the increase of accuracy of the data processing. The book contains of 54 science papers which include the results of research concerning the current directions in the fields of data mining, machine learning, and decision making. The papers are divided in terms of their topic into three sections. The first section "Analysis and Modeling of Complex Systems and Processes" contains of 26 papers, and the second section "Theoretical and Applied Aspects of Decision-Making Systems" contains of 13 papers. There are 15 papers in the third section "Computational Intelligence and Inductive Modeling". The book is focused to scientists and developers in the fields of data mining, machine learning and decision-making systems.