This IBM® Redbooks® publication addresses topics to use the virtualization strengths of the IBM POWER8® platform to solve clients' system resource utilization challenges and maximize systems' throughput and capacity. This book addresses performance tuning topics that will help answer clients' complex analytic workload requirements, help maximize systems' resources, and provide expert-level documentation to transfer the how-to-skills to the worldwide teams. This book strengthens the position of IBM Analytics and Big Data solutions with a well-defined and documented deployment model within a POWER8 virtualized environment, offering clients a planned foundation for security, scaling, capacity, resilience, and optimization for analytics workloads. This book is targeted toward technical professionals (analytics consultants, technical support staff, IT Architects, and IT Specialists) who are responsible for providing analytics solutions and support on IBM Power SystemsTM.

This IBM® Redbooks® publication addresses performance tuning topics to help leverage the virtualization strengths of the POWER® platform to solve clients' system resource utilization challenges, and maximize system throughput and capacity. We examine the performance monitoring tools, utilities, documentation, and other resources available to help technical teams provide optimized business solutions and support for applications running on IBM POWER systems' virtualized environments. The book offers application performance examples deployed on IBM Power SystemsTM utilizing performance monitoring tools to leverage the comprehensive set of POWER virtualization features: Logical Partitions (LPARs), micro-partitioning, active memory sharing, workload partitions, and more. We provide a well-defined and documented performance tuning model in a POWER system virtualized environment to help you plan a foundation for scaling, capacity, and optimization . This book targets technical professionals (technical consultants, technical support staff, IT Architects, and IT Specialists) responsible for providing solutions and support on IBM POWER systems, including performance tuning.

Data warehouses were developed for many good reasons, such as providing quick query and reporting for business operations, and business performance. However, over the years, due to the explosion of applications and data volume, many existing data warehouses have become difficult to manage. Extract, Transform, and Load (ETL) processes are taking longer, missing their allocated batch windows. In addition, data types that are required for business analysis have expanded from structured data to unstructured data. The Apache open source Hadoop platform provides a great alternative for solving these problems. IBM® has committed to open source since the early years of open Linux. IBM and Hortonworks together are committed to Apache open source software more than any other company. IBM Power SystemsTM servers are built with open technologies and are designed for mission-critical data applications. Power Systems servers use technology from the OpenPOWER Foundation, an open technology infrastructure that uses the IBM POWER® architecture to help meet the evolving needs of big data applications. The combination of Power Systems with Hortonworks Data Platform (HDP) provides users with a highly efficient platform that provides leadership performance for big data workloads such as Hadoop and Spark. This IBM RedpaperTM publication provides details about Enterprise Data Warehouse (EDW) optimization with Hadoop on Power Systems. Many people know Power Systems from the IBM AIX® platform, but might not be familiar with IBM PowerLinuxTM, so part of this paper provides a Power Systems overview. A quick introduction to Hadoop is provided for those not familiar with the topic. Details of HDP on Power Reference architecture are included that will help both software architects and infrastructure architects understand the design. In the optimization chapter, we describe various topics: traditional EDW offload, sizing guidelines, performance tuning, IBM Elastic StorageTM Server (ESS) for data-intensive workload, IBM Big SQL as the common structured query language (SQL) engine for Hadoop platform, and tools that are available on Power Systems that are related to EDW optimization. We also dedicate some pages to the analytics components (IBM Data Science Experience (IBM DSX) and IBM SpectrumTM Conductor for Spark workload) for the Hadoop infrastructure.

This IBM® Redbooks® publication focuses on gathering the correct technical information, and laying out simple guidance for optimizing code performance on IBM POWER8® processor-based systems that run the IBM AIX®, IBM i, or Linux operating systems. There is straightforward performance optimization that can be performed with a minimum of effort and without extensive previous experience or in-depth knowledge. The POWER8 processor contains many new and important performance features, such as support for eight hardware threads in each core and support for transactional memory. The POWER8 processor is a strict superset of the IBM POWER7+TM processor, and so all of the performance features of the POWER7+ processor, such as multiple page sizes, also appear in the POWER8 processor. Much of the technical information and guidance for optimizing performance on POWER8 processors that is presented in this guide also applies to POWER7+ and earlier processors, except where the guide explicitly indicates that a feature is new in the POWER8 processor. This guide strives to focus on optimizations that tend to be positive across a broad set of IBM POWER® processor chips and systems. Specific guidance is given for the POWER8 processor; however, the general guidance is applicable to the IBM POWER7+, IBM POWER7®, IBM POWER6®, IBM POWER5, and even to earlier processors. This guide is directed at personnel who are responsible for performing migration and implementation activities on POWER8 processor-based systems. This includes system administrators, system architects, network administrators, information architects, and database administrators (DBAs).

This IBM® Redbooks® publication demonstrates and documents that IBM Power SystemsTM high-performance computing and technical computing solutions deliver faster time to value with powerful solutions. Configurable into highly scalable Linux clusters, Power Systems offer extreme performance for demanding workloads such as genomics, finance, computational chemistry, oil and gas exploration, and high-performance data analytics. This book delivers a high-performance computing solution implemented on the IBM Power System S822LC. The solution delivers high application performance and throughput based on its built-for-big-data architecture that incorporates IBM POWER8® processors, tightly coupled Field Programmable Gate Arrays (FPGAs) and accelerators, and faster I/O by using Coherent Accelerator Processor Interface (CAPI). This solution is ideal for clients that need more processing power while simultaneously increasing workload density and reducing datacenter floor space requirements. The Power S822LC offers a modular design to scale from a single rack to hundreds, simplicity of ordering, and a strong innovation roadmap for graphics processing units (GPUs). This publication is targeted toward technical professionals (consultants, technical support staff, IT Architects, and IT Specialists) responsible for delivering cost effective high-performance computing (HPC) solutions that help uncover insights from their data so they can optimize business results, product development, and scientific discoveries

As big data becomes more ubiquitous, businesses are wondering how they can best leverage it to gain insight into their most important business questions. Using machine learning (ML) and deep learning (DL) in big data environments can identify historical patterns and build artificial intelligence (AI) models that can help businesses to improve customer experience, add services and offerings, identify new revenue streams or lines of business (LOBs), and optimize business or manufacturing operations. The power of AI for predictive analytics is being harnessed across all industries, so it is important that businesses familiarize themselves with all of the tools and techniques that are available for integration with their data lake environments. In this IBM® Redbooks® publication, we cover the best practices for deploying and integrating some of the best AI solutions on the market, including: IBM Watson Machine Learning Accelerator (see note for product naming) IBM Watson Studio Local IBM Power SystemsTM IBM SpectrumTM Scale IBM Data Science Experience (IBM DSX) IBM Elastic StorageTM Server Hortonworks Data Platform (HDP) Hortonworks DataFlow (HDF) H2O Driverless AI We map out all the integrations that are possible with our different AI solutions and how they can integrate with your existing or new data lake. We also walk you through some of our client use cases and show you how some of the industry leaders are using Hortonworks, IBM PowerAI, and IBM Watson Studio Local to drive decision making. We also advise you on your deployment options, when to use a GPU, and why you should use the IBM Elastic Storage Server (IBM ESS) to improve storage management. Lastly, we describe how to integrate IBM Watson Machine Learning Accelerator and Hortonworks with or without IBM Watson Studio Local, how to access real-time data, and security. Note: IBM Watson Machine Learning Accelerator is the new product name for IBM PowerAI Enterprise. Note: Hortonworks merged with Cloudera in January 2019. The new company is called Cloudera. References to Hortonworks as a business entity in this publication are now referring to the merged company. Product names beginning with Hortonworks continue to be marketed and sold under their original names.

This IBM® Redbooks® publication demonstrates and documents how to implement and manage an IBM PowerLinuxTM cluster for big data focusing on hardware management, operating systems provisioning, application provisioning, cluster readiness check, hardware, operating system, IBM InfoSphere® BigInsightsTM, IBM Platform Symphony®, IBM SpectrumTM Scale (formerly IBM GPFSTM), applications monitoring, and performance tuning. This publication shows that IBM PowerLinux clustering solutions (hardware and software) deliver significant value to clients that need cost-effective, highly scalable, and robust solutions for big data and analytics workloads. This book documents and addresses topics on how to use IBM Platform Cluster Manager to manage PowerLinux BigData data clusters through IBM InfoSphere BigInsights, Spectrum Scale, and Platform Symphony. This book documents how to set up and manage a big data cluster on PowerLinux servers to customize application and programming solutions, and to tune applications to use IBM hardware architectures. This document uses the architectural technologies and the software solutions that are available from IBM to help solve challenging technical and business problems. This book is targeted at technical professionals (consultants, technical support staff, IT Architects, and IT Specialists) that are responsible for delivering cost-effective Linux on IBM Power SystemsTM solutions that help uncover insights among client's data so they can act to optimize business results, product development, and scientific discoveries.

Today, organizations face tremendous challenges with data explosion and information governance. InfoSphereTM OptimTM solutions solve the data growth problem at the source by managing the enterprise application data. The Optim Data Growth solutions are consistent, scalable solutions that include comprehensive capabilities for managing enterprise application data across applications, databases, operating systems, and hardware platforms. You can align the management of your enterprise application data with your business objectives to improve application service levels, lower costs, and mitigate risk. In this IBM® Redbooks® publication, we describe the IBM InfoSphere Optim Data Growth solutions and a methodology that provides implementation guidance from requirements analysis through deployment and administration planning. We also discuss various implementation topics including system architecture design, sizing, scalability, security, performance, and automation. This book is intended to provide various systems development professionals, Data Solution Architects, Data Administrators, Modelers, Data Analysts, Data Integrators, or anyone who has to analyze or integrate data structures, a broad understanding about IBM InfoSphere Optim Data Growth solutions. By being used in conjunction with the product manuals and online help, this book provides guidance about implementing an optimal solution for managing your enterprise application data.

This IBM® Redbooks® publication provides topics to help the technical community take advantage of the resilience, scalability, and performance of the IBM Power SystemsTM platform to implement or integrate an IBM Data Engine for Hadoop and Spark solution for analytics solutions to access, manage, and analyze data sets to improve business outcomes. This book documents topics to demonstrate and take advantage of the analytics strengths of the IBM POWER8® platform, the IBM analytics software portfolio, and selected third-party tools to help solve customer's data analytic workload requirements. This book describes how to plan, prepare, install, integrate, manage, and show how to use the IBM Data Engine for Hadoop and Spark solution to run analytic workloads on IBM POWER8. In addition, this publication delivers documentation to complement available IBM analytics solutions to help your data analytic needs. This publication strengthens the position of IBM analytics and big data solutions with a well-defined and documented deployment model within an IBM POWER8 virtualized environment so that customers have a planned foundation for security, scaling, capacity, resilience, and optimization for analytics workloads. This book is targeted at technical professionals (analytics consultants, technical support staff, IT Architects, and IT Specialists) that are responsible for delivering analytics solutions and support on IBM Power Systems.

This IBM® Redbooks® publication addresses topics to help clients to take advantage of the virtualization strengths of the POWER® platform to solve system resource utilization challenges and maximize system throughput and capacity. This publication examines the tools, utilities, documentation, and other resources available to help technical teams provide business solutions and support for Cognos® Business Intelligence (BI) and Statistical Package for the Social Sciences (SPSS®) on Power SystemsTM virtualized environments. This book addresses topics to help address complex high availability requirements, help maximize the availability of systems, and provide expert-level documentation to the worldwide support teams. This book strengthens the position of the Cognos and SPSS solutions with a well-defined and documented deployment model within a POWER system virtualized environment. This model provides clients with a planned foundation for security, scaling, capacity, resilience, and optimization. This book is targeted toward technical professionals (BI consultants, technical support staff, IT Architects, and IT Specialists) who are responsible for providing Smart Analytics solutions and support for Cognos and SPSS on Power Systems.