This book provides an introduction to computer benchmarking. Hockney includes material concerned with the definition of performance parameters and metrics and defines a set of suitable metrics with which to measure performance and units with which to express them. He also presents new ideas resulting from the application of dimensional analysis to the field of computer benchmarking.

This book provides an introduction to computer benchmarking. Hockney includes material concerned with the definition of performance parameters and metrics and defines a set of suitable metrics with which to measure performance and units with which to express them. He also presents new ideas resulting from the application of dimensional analysis to the field of computer benchmarking. This results in the definition of a dimensionless universal scaling diagram that completely describes the scaling properties of a class of computer benchmarks on a single diagram, for all problem sizes and all computers describable by a defined set of hardware parameters.

This book serves as both a textbook and handbook on the benchmarking of systems and components used as building blocks of modern information and communication technology applications. It provides theoretical and practical foundations as well as an in-depth exploration of modern benchmarks and benchmark development. The book is divided into two parts: foundations and applications. The first part introduces the foundations of benchmarking as a discipline, covering the three fundamental elements of each benchmarking approach: metrics, workloads, and measurement methodology. The second part focuses on different application areas, presenting contributions in specific fields of benchmark development. These contributions address the unique challenges that arise in the conception and development of benchmarks for specific systems or subsystems, and demonstrate how the foundations and concepts in the first part of the book are being used in existing benchmarks. Further, the book presents a number of concrete applications and case studies based on input from leading benchmark developers from consortia such as the Standard Performance Evaluation Corporation (SPEC) and the Transaction Processing Performance Council (TPC). Providing both practical and theoretical foundations, as well as a detailed discussion of modern benchmarks and their development, the book is intended as a handbook for professionals and researchers working in areas related to benchmarking. It offers an up-to-date point of reference for existing work as well as latest results, research challenges, and future research directions. It also can be used as a textbook for graduate and postgraduate students studying any of the many subjects related to benchmarking. While readers are assumed to be familiar with the principles and practices of computer science, as well as software and systems engineering, no specific expertise in any subfield of these disciplines is required.

This book constitutes the thoroughly refereed proceedings of the 5th International Workshop, PMBS 2014 in New Orleans, LA, USA in November 2014. The 12 full and 2 short papers presented in this volume were carefully reviewed and selected from 53 submissions. The papers cover topics on performance benchmarking and optimization; performance analysis and prediction; and power, energy and checkpointing.

First established in August 1988, the Transaction Processing Performance Council (TPC) has shaped the landscape of modern transaction processing and database benchmarks over two decades. Now, the world is in the midst of an extraordinary information explosion led by rapid growth in the use of the Internet and connected devices. Both user-generated data and enterprise data levels continue to grow ex- nentially. With substantial technological breakthroughs, Moore's law will continue for at least a decade, and the data storage capacities and data transfer speeds will continue to increase exponentially. These have challenged industry experts and researchers to develop innovative techniques to evaluate and benchmark both hardware and software technologies. As a result, the TPC held its First Conference on Performance Evaluation and Benchmarking (TPCTC 2009) on August 24 in Lyon, France in conjunction with the 35th International Conference on Very Large Data Bases (VLDB 2009). TPCTC 2009 provided industry experts and researchers with a forum to present and debate novel ideas and methodologies in performance evaluation, measurement and characteri- tion for 2010 and beyond. This book contains the proceedings of this conference, including 16 papers and keynote papers from Michael Stonebraker and Karl Huppler.

Peer-to-peer systems are now widely used and have become the focus of attention for many researchers over the past decade. A number of algorithms for decentralized search, content distribution, and media streaming have been developed. This book provides fundamental concepts for the benchmarking of those algorithms in peer-to-peer systems. It also contains a collection of characteristic benchmarking results. The chapters of the book have been organized in three topical sections on: Fundamentals of Benchmarking in P2P Systems; Synthetic Benchmarks for Peer-to-Peer Systems; and Application Benchmarks for Peer-to-Peer Systems. They are preceded by a detailed introduction to the subject.

Computer Architecture/Software Engineering

This book constitutes the refereed post-conference proceedings of the 12th TPC Technology Conference on Performance Evaluation and Benchmarking, TPCTC 2020, held in August 2020.The 8 papers presented were carefully reviewed and cover the following topics: testing ACID compliance in the LDBC social network benchmark; experimental performance evaluation of stream processing engines made easy; revisiting issues in benchmarking metric selection; performance evaluation for digital transformation; experimental comparison of relational and NoSQL document systems; a framework for supporting repetition and evaluation in the process of cloud-based DBMS performance benchmarking; benchmarking AI inference; a domain independent benchmark evolution model for the transaction processing performance council.

Provide evidence-based answers that can be measured and relied upon by your business. Database administrators will be able to make sound architectural decisions in a fast-changing landscape of virtualized servers and container-based solutions based on the empirical method presented in this book for answering “what if” questions about database performance. Today’s database administrators face numerous questions such as: What if we consolidate databases using multitenant features? What if we virtualize database servers as Docker containers? What if we deploy the latest in NVMe flash disks to speed up IO access? Do features such as compression, partitioning, and in-memory OLTP earn back their price? What if we move our databases to the cloud? As an administrator, do you know the answers or even how to test the assumptions? Database Benchmarking and Stress Testing introduces you to database benchmarking using industry-standard test suites such as the TCP series of benchmarks, which are the same benchmarks that vendors rely upon. You’ll learn to run these industry-standard benchmarks and collect results to use in answering questions about the performance impact of architectural changes, technology changes, and even down to the brand of database software. You’ll learn to measure performance and predict the specific impact of changes to your environment. You’ll know the limitations of the benchmarks and the crucial difference between benchmarking and workload capture/reply. This book teaches you how to create empirical evidence in support of business and technology decisions. It’s about not guessing when you should be measuring. Empirical testing is scientific testing that delivers measurable results. Begin with a hypothesis about the impact of a possible architecture or technology change. Then run the appropriate benchmarks to gather data and predict whether the change you’re exploring will be beneficial, and by what order of magnitude. Stop guessing. Start measuring. Let Database Benchmarking and Stress Testing show the way. What You'll LearnUnderstand the industry-standard database benchmarks, and when each is best used Prepare for a database benchmarking effort so reliable results can be achieved Perform database benchmarking for consolidation, virtualization, and cloud projects Recognize and avoid common mistakes in benchmarking database performanceMeasure and interpret results in a rational, concise manner for reliable comparisonsChoose and provide advice on benchmarking tools based on their pros and cons Who This Book Is For Database administrators and professionals responsible for advising on architectural decisions such as whether to use cloud-based services, whether to consolidate and containerize, and who must make recommendations on storage or any other technology that impacts database performance

In its fourth edition, this book focuses on real-world examples and practical applications and encourages students to develop a "big-picture" understanding of how essential organization and architecture concepts are applied in the computing world. In addition to direct correlation with the ACM/IEEE CS2013 guidelines for computer organization and architecture, the text exposes readers to the inner workings of a modern digital computer through an integrated presentation of fundamental concepts and principles. It includes the most up-to-the-minute data and resources available and reflects current technologies, including tablets and cloud computing. All-new exercises, expanded discussions, and feature boxes in every chapter implement even more real-world applications and current data, and many chapters include all-new examples. --