Preconditioning and the Conjugate Gradient Method in the Context of Solving PDEs?is about the interplay between modeling, analysis, discretization, matrix computation, and model reduction. The authors link PDE analysis, functional analysis, and calculus of variations with matrix iterative computation using Krylov subspace methods and address the challenges that arise during formulation of the mathematical model through to efficient numerical solution of the algebraic problem. The book?s central concept, preconditioning of the conjugate gradient method, is traditionally developed algebraically using the preconditioned finite-dimensional algebraic system. In this text, however, preconditioning is connected to the PDE analysis, and the infinite-dimensional formulation of the conjugate gradient method and its discretization and preconditioning are linked together. This text challenges commonly held views, addresses widespread misunderstandings, and formulates thought-provoking open questions for further research.?

Two approaches are known for solving large-scale unconstrained optimization problems—the limited-memory quasi-Newton method (truncated Newton method) and the conjugate gradient method. This is the first book to detail conjugate gradient methods, showing their properties and convergence characteristics as well as their performance in solving large-scale unconstrained optimization problems and applications. Comparisons to the limited-memory and truncated Newton methods are also discussed. Topics studied in detail include: linear conjugate gradient methods, standard conjugate gradient methods, acceleration of conjugate gradient methods, hybrid, modifications of the standard scheme, memoryless BFGS preconditioned, and three-term. Other conjugate gradient methods with clustering the eigenvalues or with the minimization of the condition number of the iteration matrix, are also treated. For each method, the convergence analysis, the computational performances and the comparisons versus other conjugate gradient methods are given. The theory behind the conjugate gradient algorithms presented as a methodology is developed with a clear, rigorous, and friendly exposition; the reader will gain an understanding of their properties and their convergence and will learn to develop and prove the convergence of his/her own methods. Numerous numerical studies are supplied with comparisons and comments on the behavior of conjugate gradient algorithms for solving a collection of 800 unconstrained optimization problems of different structures and complexities with the number of variables in the range [1000,10000]. The book is addressed to all those interested in developing and using new advanced techniques for solving unconstrained optimization complex problems. Mathematical programming researchers, theoreticians and practitioners in operations research, practitioners in engineering and industry researchers, as well as graduate students in mathematics, Ph.D. and master students in mathematical programming, will find plenty of information and practical applications for solving large-scale unconstrained optimization problems and applications by conjugate gradient methods.

This book details algorithms for large-scale unconstrained and bound constrained optimization. It shows optimization techniques from a conjugate gradient algorithm perspective as well as methods of shortest residuals, which have been developed by the author.

The position taken in this collection of pedagogically written essays is that conjugate gradient algorithms and finite element methods complement each other extremely well. Via their combinations practitioners have been able to solve complicated, direct and inverse, multidemensional problems modeled by ordinary or partial differential equations and inequalities, not necessarily linear, optimal control and optimal design being part of these problems. The aim of this book is to present both methods in the context of complicated problems modeled by linear and nonlinear partial differential equations, to provide an in-depth discussion on their implementation aspects. The authors show that conjugate gradient methods and finite element methods apply to the solution of real-life problems. They address graduate students as well as experts in scientific computing.

Mathematics of Computing -- Numerical Analysis.

Finite Element Solution of Boundary Value Problems: Theory and Computation provides an introduction to both the theoretical and computational aspects of the finite element method for solving boundary value problems for partial differential equations. This book is composed of seven chapters and begins with surveys of the two kinds of preconditioning techniques, one based on the symmetric successive overrelaxation iterative method for solving a system of equations and a form of incomplete factorization. The subsequent chapters deal with the concepts from functional analysis of boundary value problems. These topics are followed by discussions of the Ritz method, which minimizes the quadratic functional associated with a given boundary value problem over some finite-dimensional subspace of the original space of functions. Other chapters are devoted to direct methods, including Gaussian elimination and related methods, for solving a system of linear algebraic equations. The final chapter continues the analysis of preconditioned conjugate gradient methods, concentrating on applications to finite element problems. This chapter also looks into the techniques for reducing rounding errors in the iterative solution of finite element equations. This book will be of value to advanced undergraduates and graduates in the areas of numerical analysis, mathematics, and computer science, as well as for theoretically inclined workers in engineering and the physical sciences.

The most comprehensive and up-to-date discussion available of the Lanczos and CG methods for computing eigenvalues and solving linear systems.

After reading this book, students should be able to analyze computational problems in linear algebra such as linear systems, least squares- and eigenvalue problems, and to develop their own algorithms for solving them. Since these problems can be large and difficult to handle, much can be gained by understanding and taking advantage of special structures. This in turn requires a good grasp of basic numerical linear algebra and matrix factorizations. Factoring a matrix into a product of simpler matrices is a crucial tool in numerical linear algebra, because it allows us to tackle complex problems by solving a sequence of easier ones. The main characteristics of this book are as follows: It is self-contained, only assuming that readers have completed first-year calculus and an introductory course on linear algebra, and that they have some experience with solving mathematical problems on a computer. The book provides detailed proofs of virtually all results. Further, its respective parts can be used independently, making it suitable for self-study. The book consists of 15 chapters, divided into five thematically oriented parts. The chapters are designed for a one-week-per-chapter, one-semester course. To facilitate self-study, an introductory chapter includes a brief review of linear algebra.