Abstract. Previous analyses of pattern search algorithms for unconstrained and linearly constrained minimization have focused on proving convergence of a subsequence of iterates to a limit point satisfying either directional or first-order necessary conditions for optimality, depending on the smoothness of the objective function in a neighborhood of the limit point. Even though pattern search methods require no derivative information, we are able to prove some limited directional second-order results. Although not as strong as classical second-order necessary conditions, these results are stronger than the first order conditions that many gradient-based methods satisfy. Under fairly mild conditions, we can eliminate from consideration all strict local maximizers and an entire class of saddle points.

The book is intended for graduate students and researchers in mathematics, computer science, and operational research. The book presents a new derivative-free optimization method/algorithm based on randomly generated trial points in specified domains and where the best ones are selected at each iteration by using a number of rules. This method is different from many other well established methods presented in the literature and proves to be competitive for solving many unconstrained optimization problems with different structures and complexities, with a relative large number of variables. Intensive numerical experiments with 140 unconstrained optimization problems, with up to 500 variables, have shown that this approach is efficient and robust. Structured into 4 chapters, Chapter 1 is introductory. Chapter 2 is dedicated to presenting a two level derivative-free random search method for unconstrained optimization. It is assumed that the minimizing function is continuous, lower bounded and its minimum value is known. Chapter 3 proves the convergence of the algorithm. In Chapter 4, the numerical performances of the algorithm are shown for solving 140 unconstrained optimization problems, out of which 16 are real applications. This shows that the optimization process has two phases: the reduction phase and the stalling one. Finally, the performances of the algorithm for solving a number of 30 large-scale unconstrained optimization problems up to 500 variables are presented. These numerical results show that this approach based on the two level random search method for unconstrained optimization is able to solve a large diversity of problems with different structures and complexities. There are a number of open problems which refer to the following aspects: the selection of the number of trial or the number of the local trial points, the selection of the bounds of the domains where the trial points and the local trial points are randomly generated and a criterion for initiating the line search.

This book is designed as a textbook, suitable for self-learning or for teaching an upper-year university course on derivative-free and blackbox optimization. The book is split into 5 parts and is designed to be modular; any individual part depends only on the material in Part I. Part I of the book discusses what is meant by Derivative-Free and Blackbox Optimization, provides background material, and early basics while Part II focuses on heuristic methods (Genetic Algorithms and Nelder-Mead). Part III presents direct search methods (Generalized Pattern Search and Mesh Adaptive Direct Search) and Part IV focuses on model-based methods (Simplex Gradient and Trust Region). Part V discusses dealing with constraints, using surrogates, and bi-objective optimization. End of chapter exercises are included throughout as well as 15 end of chapter projects and over 40 figures. Benchmarking techniques are also presented in the appendix.

Numerical minimization of an objective function is analyzed in this book to understand solution algorithms for optimization problems. Multiset-mappings are introduced to engineer numerical minimization as a repeated application of an iteration mapping. Ideas from numerical variational analysis are extended to define and explore notions of continuity and differentiability of multiset-mappings, and prove a fixed-point theorem for iteration mappings. Concepts from dynamical systems are utilized to develop notions of basin size and basin entropy. Simulations to estimate basins of attraction, to measure and classify basin size, and to compute basin are included to shed new light on convergence behavior in numerical minimization. Graduate students, researchers, and practitioners in optimization and mathematics who work theoretically to develop solution algorithms will find this book a useful resource.

This volume consists of chapters written by eminent scientists and engineers from the international community and present significant advances in several theories, methods and applications of an interdisciplinary research. These contributions focus on both old and recent developments of Global Optimization Theory, Convex Analysis, Calculus of Variations, Discrete Mathematics and Geometry, as well as several applications to a large variety of concrete problems, including applications of computers to the study of smoothness and analyticity of functions, applications to epidemiological diffusion, networks, mathematical models of elastic and piezoelectric fields, optimal algorithms, stability of neutral type vector functional differential equations, sampling and rational interpolation for non-band-limited signals, recurrent neural network for convex optimization problems and experimental design. The book also contains some review works, which could prove particularly useful for a broader audience of readers in Mathematical and Engineering subjects and especially to graduate students who search for the latest information.

In this landmark volume, J. Rodgers Hollingsworth, Karl H. M ller, and Ellen Jane Hollingsworth take a first step towards imposing order on the increasingly diverse field of socio-economics by embedding the various disciplines and sub-disciplines in a common core. The distinguished contributors in this volume show how institutions, governance arrangements, societal sectors, organizations, individual actors, and innovativeness are intertwined and, ultimately, how individuals and firms have a high degree of autonomy. By offering original suggestions and guidelines for developing a socio-economics research agenda focused on institutional analysis, Advancing Socio-Economics: An Institutionalist Perspective, will enlighten all interested in the social sciences.

Optimization is of critical importance in engineering. Engineers constantly strive for the best possible solutions, the most economical use of limited resources, and the greatest efficiency. As system complexity increases, these goals mandate the use of state-of-the-art optimization techniques. In recent years, the theory and methodology of optimization have seen revolutionary improvements. Moreover, the exponential growth in computational power, along with the availability of multicore computing with virtually unlimited memory and storage capacity, has fundamentally changed what engineers can do to optimize their designs. This is a two-way process: engineers benefit from developments in optimization methodology, and challenging new classes of optimization problems arise from novel engineering applications. Advances and Trends in Optimization with Engineering Applications reviews 10 major areas of optimization and related engineering applications, providing a broad summary of state-of-the-art optimization techniques most important to engineering practice. Each part provides a clear overview of a specific area and discusses a range of real-world problems. The book provides a solid foundation for engineers and mathematical optimizers alike who want to understand the importance of optimization methods to engineering and the capabilities of these methods.

Recently, novel metaheuristic techniques have emerged in response to the limitations of conventional approaches, leading to enhanced outcomes. These new methods introduce interesting mechanisms and innovative collaborative strategies that facilitate the efficient exploration and exploitation of extensive search spaces characterized by numerous dimensions. The objective of this book is to present advancements that discuss novel alternative metaheuristic developments that have demonstrated their effectiveness in tackling various complex problems. This book encompasses a variety of emerging metaheuristic methods and their practical applications. The content is presented from a teaching perspective, making it particularly suitable for undergraduate and postgraduate students in fields such as science, electrical engineering, and computational mathematics. The book aligns well with courses in artificial intelligence, electrical engineering, and evolutionary computation. Furthermore, the material offers valuable insights to researchers within the metaheuristic and engineering communities. Similarly, engineering practitioners unfamiliar with metaheuristic computation concepts will recognize the pragmatic value of the discussed techniques. These methods transcend mere theoretical tools that have been adapted to effectively address the significant real-world problems commonly encountered in engineering domains.