arise automatically as a result of the recursive structure of the task and the continuous nature of the SRN's state space. Elman also introduces a new graphical technique for study ing network behavior based on principal components analysis. He shows that sentences with multiple levels of embedding produce state space trajectories with an intriguing self similar structure. The development and shape of a recurrent network's state space is the subject of Pollack's paper, the most provocative in this collection. Pollack looks more closely at a connectionist network as a continuous dynamical system. He describes a new type of machine learning phenomenon: induction by phase transition. He then shows that under certain conditions, the state space created by these machines can have a fractal or chaotic structure, with a potentially infinite number of states. This is graphically illustrated using a higher-order recurrent network trained to recognize various regular languages over binary strings. Finally, Pollack suggests that it might be possible to exploit the fractal dynamics of these systems to achieve a generative capacity beyond that of finite-state machines.

Modern research in neural networks has led to powerful artificial learning systems, while recent work in the psychology of human memory has revealed much about how natural systems really learn, including the role of unconscious, implicit, memory processes. Regrettably, the two approaches typically ignore each other. This book, combining the approaches, should contribute to their mutual benefit. New empirical work is presented showing dissociations between implicit and explicit memory performance. Recently proposed explanations for such data lead to a new connectionist learning procedure: CALM (Categorizing and Learning Module), which can learn with or without supervision, and shows practical advantages over many existing procedures. Specific experiments are simulated by a network model (ELAN) composed of CALM modules. A working memory extension to the model is also discussed that could give it symbol manipulation abilities. The book will be of interest to memory psychologists and connectionists, as well as to cognitive scientists who in the past have tended to restrict themselves to symbolic models.

Using the tools of complexity theory, Stephen Judd develops a formal description of associative learning in connectionist networks. He rigorously exposes the computational difficulties in training neural networks and explores how certain design principles will or will not make the problems easier.Judd looks beyond the scope of any one particular learning rule, at a level above the details of neurons. There he finds new issues that arise when great numbers of neurons are employed and he offers fresh insights into design principles that could guide the construction of artificial and biological neural networks.The first part of the book describes the motivations and goals of the study and relates them to current scientific theory. It provides an overview of the major ideas, formulates the general learning problem with an eye to the computational complexity of the task, reviews current theory on learning, relates the book's model of learning to other models outside the connectionist paradigm, and sets out to examine scale-up issues in connectionist learning.Later chapters prove the intractability of the general case of memorizing in networks, elaborate on implications of this intractability and point out several corollaries applying to various special subcases. Judd refines the distinctive characteristics of the difficulties with families of shallow networks, addresses concerns about the ability of neural networks to generalize, and summarizes the results, implications, and possible extensions of the work. Neural Network Design and the Complexity of Learning is included in the Network Modeling and Connectionism series edited by Jeffrey Elman.

Addressing the current tension within the artificial intelligence community betweenadvocates of powerful symbolic representations that lack efficient learning procedures and advocatesof relatively simple learning procedures that lack the ability to represent complex structureseffectively.

Explains what connectionist learning is and how it relates to artificial intelligence. Develops a respresentation of knowledge and a representation of a simple computational system, and gives some examples of how such a system might work.

This second edition of the must-read work in the field presents generic computational models and techniques that can be used for the development of evolving, adaptive modeling systems, as well as new trends including computational neuro-genetic modeling and quantum information processing related to evolving systems. New applications, such as autonomous robots, adaptive artificial life systems and adaptive decision support systems are also covered.

This book presents a fascinating and self-contained account of "recruitment learning", a model and theory of fast learning in the neocortex. In contrast to the more common attractor network paradigm for long- and short-term memory, recruitment learning focuses on one-shot learning or "chunking" of arbitrary feature conjunctions that co-occur in single presentations. The book starts with a comprehensive review of the historic background of recruitment learning, putting special emphasis on the ground-breaking work of D.O. Hebb, W.A.Wickelgren, J.A.Feldman, L.G.Valiant, and L. Shastri. Afterwards a thorough mathematical analysis of the model is presented which shows that recruitment is indeed a plausible mechanism of memory formation in the neocortex. A third part extends the main concepts towards state-of-the-art spiking neuron models and dynamic synchronization as a tentative solution of the binding problem. The book further discusses the possible role of adult neurogenesis for recruitment. These recent developments put the theory of recruitment learning at the forefront of research on biologically inspired memory models and make the book an important and timely contribution to the field.

Presenting research on the computational abilities of connectionist, neural, and neurally inspired systems, this series emphasizes the question of how connectionist or neural network models can be made to perform rapid, short-term types of computation that are useful in higher level cognitive processes. The most recent volumes are directed mainly at researchers in connectionism, analogy, metaphor, and case-based reasoning, but are also suitable for graduate courses in those areas.

In the "black box function optimization" problem, a search strategy is required to find an extremal point of a function without knowing the structure of the function or the range of possible function values. Solving such problems efficiently requires two abilities. On the one hand, a strategy must be capable of learning while searching: It must gather global information about the space and concentrate the search in the most promising regions. On the other hand, a strategy must be capable of sustained exploration: If a search of the most promising region does not uncover a satisfactory point, the strategy must redirect its efforts into other regions of the space. This dissertation describes a connectionist learning machine that produces a search strategy called stochastic iterated genetic hillclimb ing (SIGH). Viewed over a short period of time, SIGH displays a coarse-to-fine searching strategy, like simulated annealing and genetic algorithms. However, in SIGH the convergence process is reversible. The connectionist implementation makes it possible to diverge the search after it has converged, and to recover coarse-grained informa tion about the space that was suppressed during convergence. The successful optimization of a complex function by SIGH usually in volves a series of such converge/diverge cycles.