How do managers and entrepreneurs evaluate risk, encourage creativityor manage change? Might a better grasp of science help? The authorsof this book suggest that there is real value in trying to connectscience to business and that science is far too important just to beleft to the scientists

The authors suggest that a clearer understanding of entropy and the choices it presents will assist in management of change--or, as they put it, to manage disorder one needs to control the entropy vector.

How do managers and entrepreneurs evaluate risk, encourage creativity or manage change? Might a better grasp of science help? The authors of this book suggest that there is real value in trying to connect science to business and that science is far too important just to be left to the scientists.All of science is too large a prospect, so the authors limit themselves to looking at disorder. We must all learn to manage and control change, and there is plenty of social, technical and business change going on. The authors suggest that a clearer understanding of entropy and the choices it presents will assist in that management of change — or, as they put it, to manage disorder one needs to control the entropy vector.This book is for scientists and engineers aspiring to business success and for business people interested in new approaches.

Scientific knowledge grows at a phenomenal pace--but few books have had as lasting an impact or played as important a role in our modern world as The Mathematical Theory of Communication, published originally as a paper on communication theory more than fifty years ago. Republished in book form shortly thereafter, it has since gone through four hardcover and sixteen paperback printings. It is a revolutionary work, astounding in its foresight and contemporaneity. The University of Illinois Press is pleased and honored to issue this commemorative reprinting of a classic.

This monograph offers a new foundation for information theory that is based on the notion of information-as-distinctions, being directly measured by logical entropy, and on the re-quantification as Shannon entropy, which is the fundamental concept for the theory of coding and communications. Information is based on distinctions, differences, distinguishability, and diversity. Information sets are defined that express the distinctions made by a partition, e.g., the inverse-image of a random variable so they represent the pre-probability notion of information. Then logical entropy is a probability measure on the information sets, the probability that on two independent trials, a distinction or “dit” of the partition will be obtained. The formula for logical entropy is a new derivation of an old formula that goes back to the early twentieth century and has been re-derived many times in different contexts. As a probability measure, all the compound notions of joint, conditional, and mutual logical entropy are immediate. The Shannon entropy (which is not defined as a measure in the sense of measure theory) and its compound notions are then derived from a non-linear dit-to-bit transform that re-quantifies the distinctions of a random variable in terms of bits—so the Shannon entropy is the average number of binary distinctions or bits necessary to make all the distinctions of the random variable. And, using a linearization method, all the set concepts in this logical information theory naturally extend to vector spaces in general—and to Hilbert spaces in particular—for quantum logical information theory which provides the natural measure of the distinctions made in quantum measurement. Relatively short but dense in content, this work can be a reference to researchers and graduate students doing investigations in information theory, maximum entropy methods in physics, engineering, and statistics, and to all those with a special interest in a new approach to quantum information theory.

Information theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-correction. Inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations, are developed alongside applications to clustering, convolutional codes, independent component analysis, and neural networks. Uniquely, the book covers state-of-the-art error-correcting codes, including low-density-parity-check codes, turbo codes, and digital fountain codes - the twenty-first-century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, the book is ideal for self-learning, and for undergraduate or graduate courses. It also provides an unparalleled entry point for professionals in areas as diverse as computational biology, financial engineering and machine learning.

This book is a printed edition of the Special Issue "Transfer Entropy" that was published in Entropy

Entropy – the key concept of thermodynamics, clearly explained and carefully illustrated. This book presents an accurate definition of entropy in classical thermodynamics which does not “put the cart before the horse” and is suitable for basic and advanced university courses in thermodynamics. Entropy is the most important and at the same time the most difficult term of thermodynamics to understand. Many students are discontent with its classical definition since it is either based on “temperature” and “heat” which both cannot be accurately defined without entropy, or since it includes concepts such as “molecular disorder” which does not fit in a macroscopic theory. The physicists Elliott Lieb and Jakob Yngvason have recently developed a new formulation of thermodynamics which is free of these problems. The Lieb-Yngvason formulation of classical thermodynamics is based on the concept of adiabatic accessibility and culminates in the entropy principle. The entropy principle represents the accurate mathematical formulation of the second law of thermodynamics. Temperature becomes a derived quantity whereas ”heat” is no longer needed. This book makes the Lieb-Yngvason theory accessible to students. The presentation is supplemented by seven illustrative examples which explain the application of entropy and the entropy principle in practical problems in science and engineering.

Every thought is a throw of dice. Stephane Mallarme This book is the last one of a trilogy which reports a part of our research work over nearly thirty years (we discard our non-conventional results in automatic control theory and applications on the one hand, and fuzzy sets on the other), and its main key words are Information Theory, Entropy, Maximum Entropy Principle, Linguistics, Thermodynamics, Quantum Mechanics, Fractals, Fractional Brownian Motion, Stochastic Differential Equations of Order n, Stochastic Optimal Control, Computer Vision. Our obsession has been always the same: Shannon's information theory should play a basic role in the foundations of sciences, but subject to the condition that it be suitably generalized to allow us to deal with problems which are not necessarily related to communication engineering. With this objective in mind, two questions are of utmost importance: (i) How can we introduce meaning or significance of information in Shannon's information theory? (ii) How can we define and/or measure the amount of information involved in a form or a pattern without using a probabilistic scheme? It is obligatory to find suitable answers to these problems if we want to apply Shannon's theory to science with some chance of success. For instance, its use in biology has been very disappointing, for the very reason that the meaning of information is there of basic importance, and is not involved in this approach.