The last two decades have witnessed an enormous growth with regard to ap plications of information theoretic framework in areas of physical, biological, engineering and even social sciences. In particular, growth has been spectac ular in the field of information technology,soft computing,nonlinear systems and molecular biology. Claude Shannon in 1948 laid the foundation of the field of information theory in the context of communication theory. It is in deed remarkable that his framework is as relevant today as was when he 1 proposed it. Shannon died on Feb 24, 2001. Arun Netravali observes "As if assuming that inexpensive, high-speed processing would come to pass, Shan non figured out the upper limits on communication rates. First in telephone channels, then in optical communications, and now in wireless, Shannon has had the utmost value in defining the engineering limits we face". Shannon introduced the concept of entropy. The notable feature of the entropy frame work is that it enables quantification of uncertainty present in a system. In many realistic situations one is confronted only with partial or incomplete information in the form of moment, or bounds on these values etc. ; and it is then required to construct a probabilistic model from this partial information. In such situations, the principle of maximum entropy provides a rational ba sis for constructing a probabilistic model. It is thus necessary and important to keep track of advances in the applications of maximum entropy principle to ever expanding areas of knowledge.

This senior-level textbook on entropy provides a conceptual framework for the study of probabilistic systems with its elucidation of three key concepts - Shannon's information theory, Jaynes' maximum entropy principle and Kullback's minimum cross-entropy principle.

Entropy optimization is a useful combination of classical engineering theory (entropy) with mathematical optimization. The resulting entropy optimization models have proved their usefulness with successful applications in areas such as image reconstruction, pattern recognition, statistical inference, queuing theory, spectral analysis, statistical mechanics, transportation planning, urban and regional planning, input-output analysis, portfolio investment, information analysis, and linear and nonlinear programming. While entropy optimization has been used in different fields, a good number of applicable solution methods have been loosely constructed without sufficient mathematical treatment. A systematic presentation with proper mathematical treatment of this material is needed by practitioners and researchers alike in all application areas. The purpose of this book is to meet this need. Entropy Optimization and Mathematical Programming offers perspectives that meet the needs of diverse user communities so that the users can apply entropy optimization techniques with complete comfort and ease. With this consideration, the authors focus on the entropy optimization problems in finite dimensional Euclidean space such that only some basic familiarity with optimization is required of the reader.

Entropy Theory and its Application in Environmental and Water Engineering responds to the need for a book that deals with basic concepts of entropy theory from a hydrologic and water engineering perspective and then for a book that deals with applications of these concepts to a range of water engineering problems. The range of applications of entropy is constantly expanding and new areas finding a use for the theory are continually emerging. The applications of concepts and techniques vary across different subject areas and this book aims to relate them directly to practical problems of environmental and water engineering. The book presents and explains the Principle of Maximum Entropy (POME) and the Principle of Minimum Cross Entropy (POMCE) and their applications to different types of probability distributions. Spatial and inverse spatial entropy are important for urban planning and are presented with clarity. Maximum entropy spectral analysis and minimum cross entropy spectral analysis are powerful techniques for addressing a variety of problems faced by environmental and water scientists and engineers and are described here with illustrative examples. Giving a thorough introduction to the use of entropy to measure the unpredictability in environmental and water systems this book will add an essential statistical method to the toolkit of postgraduates, researchers and academic hydrologists, water resource managers, environmental scientists and engineers. It will also offer a valuable resource for professionals in the same areas, governmental organizations, private companies as well as students in earth sciences, civil and agricultural engineering, and agricultural and rangeland sciences. This book: Provides a thorough introduction to entropy for beginners and more experienced users Uses numerous examples to illustrate the applications of the theoretical principles Allows the reader to apply entropy theory to the solution of practical problems Assumes minimal existing mathematical knowledge Discusses the theory and its various aspects in both univariate and bivariate cases Covers newly expanding areas including neural networks from an entropy perspective and future developments.

Since the landmark contributions of C. E. Shannon in 1948, and those of E. T. Jaynes about a decade later, applications of the concept of entropy and the principle of maximum entropy have proliterated in science and engineering. Recent years have witnessed a broad range of new and exciting developments in hydrology and water resources using the entropy concept. These have encompassed innovative methods for hydrologic network design, transfer of information, flow forecasting, reliability assessment for water distribution systems, parameter estimation, derivation of probability distributions, drainage-network analysis, sediment yield modeling and pollutant loading, bridge-scour analysis, construction of velocity profiles, comparative evaluation of hydrologic models, and so on. Some of these methods hold great promise for advancement of engineering practice, permitting rational alternatives to conventional approaches. On the other hand, the concepts of energy and energy dissipation are being increasingly applied to a wide spectrum of problems in environmental and water resources. Both entropy and energy dissipation have their origin in thermodynamics, and are related concepts. Yet, many of the developments using entropy seem to be based entirely on statistical interpretation and have seemingly little physical content. For example, most of the entropy-related developments and applications in water resources have been based on the information-theoretic interpretation of entropy. We believe if the power of the entropy concept is to be fully realized, then its physical basis has to be established.

This book shows the potential of entropy and information theory in forecasting, including both theoretical developments and empirical applications. The contents cover a great diversity of topics, such as the aggregation and combination of individual forecasts, the comparison of forecasting performance, and the debate concerning the tradeoff between complexity and accuracy. Analyses of forecasting uncertainty, robustness, and inconsistency are also included, as are proposals for new forecasting approaches. The proposed methods encompass a variety of time series techniques (e.g., ARIMA, VAR, state space models) as well as econometric methods and machine learning algorithms. The empirical contents include both simulated experiments and real-world applications focusing on GDP, M4-Competition series, confidence and industrial trend surveys, and stock exchange composite indices, among others. In summary, this collection provides an engaging insight into entropy applications for forecasting, offering an interesting overview of the current situation and suggesting possibilities for further research in this field.

In this monograph, a statistical description of natural phenomena is used to develop an information processing system capable of modeling non-linear relationships between sensory data. The system, based on self-organized, optimal preservation of empirical information, applies these relationships for prediction and adaptive control. This monograph is written for students, scientists and engineers in academia and industry who are interested in experimental work related to the adaptive modeling of natural laws, the development of sensory-neural networks, intelligent control, synergetics and informatics. No specific knowledge of advanced mathematics is presupposed. Examples taken from physics, engineering, medicine and economics demonstrate the applicability of such intelligent systems.

Dictionary of Scientific Principles presents a unique and timeless collection of (almost) all known rules or laws commonly called principles, identified throughout the history of scientific development, their definition, and use. Exploring a broad range of disciplines, the book first lists more than 2,000 principles organized in a standard alphabetical order, then provides a list of subject headings for which related principles are identified. A staple addition to every library, the dictionary will also be of interest to scientists and general readers.

Increasing demand on improving the resiliency of modern structures and infrastructure requires ever more critical and complex designs. Therefore, the need for accurate and efficient approaches to assess uncertainties in loads, geometry, material properties, manufacturing processes, and operational environments has increased significantly. Reliability-based techniques help develop more accurate initial guidance for robust design and help to identify the sources of significant uncertainty in structural systems. Reliability-Based Analysis and Design of Structures and Infrastructure presents an overview of the methods of classical reliability analysis and design most associated with structural reliability. It also introduces more modern methods and advancements, and emphasizes the most useful methods and techniques used in reliability and risk studies, while elaborating their practical applications and limitations rather than detailed derivations. Features: Provides a practical and comprehensive overview of reliability and risk analysis and design techniques. Introduces resilient and smart structures/infrastructure that will lead to more reliable and sustainable societies. Considers loss elimination, risk management and life-cycle asset management as related to infrastructure projects. Introduces probability theory, statistical methods, and reliability analysis methods. Reliability-Based Analysis and Design of Structures and Infrastructure is suitable for researchers and practicing engineers, as well as upper-level students taking related courses in structural reliability analysis and design.

This accessible and engaging textbook presents a concise introduction to the exciting field of artificial intelligence (AI). The broad-ranging discussion covers the key subdisciplines within the field, describing practical algorithms and concrete applications in the areas of agents, logic, search, reasoning under uncertainty, machine learning, neural networks, and reinforcement learning. Fully revised and updated, this much-anticipated second edition also includes new material on deep learning. Topics and features: presents an application-focused and hands-on approach to learning, with supplementary teaching resources provided at an associated website; contains numerous study exercises and solutions, highlighted examples, definitions, theorems, and illustrative cartoons; includes chapters on predicate logic, PROLOG, heuristic search, probabilistic reasoning, machine learning and data mining, neural networks and reinforcement learning; reports on developments in deep learning, including applications of neural networks to generate creative content such as text, music and art (NEW); examines performance evaluation of clustering algorithms, and presents two practical examples explaining Bayes’ theorem and its relevance in everyday life (NEW); discusses search algorithms, analyzing the cycle check, explaining route planning for car navigation systems, and introducing Monte Carlo Tree Search (NEW); includes a section in the introduction on AI and society, discussing the implications of AI on topics such as employment and transportation (NEW). Ideal for foundation courses or modules on AI, this easy-to-read textbook offers an excellent overview of the field for students of computer science and other technical disciplines, requiring no more than a high-school level of knowledge of mathematics to understand the material.