Perspectives on Data Science for Software Engineering presents the best practices of seasoned data miners in software engineering. The idea for this book was created during the 2014 conference at Dagstuhl, an invitation-only gathering of leading computer scientists who meet to identify and discuss cutting-edge informatics topics. At the 2014 conference, the concept of how to transfer the knowledge of experts from seasoned software engineers and data scientists to newcomers in the field highlighted many discussions. While there are many books covering data mining and software engineering basics, they present only the fundamentals and lack the perspective that comes from real-world experience. This book offers unique insights into the wisdom of the community's leaders gathered to share hard-won lessons from the trenches. Ideas are presented in digestible chapters designed to be applicable across many domains. Topics included cover data collection, data sharing, data mining, and how to utilize these techniques in successful software projects. Newcomers to software engineering data science will learn the tips and tricks of the trade, while more experienced data scientists will benefit from war stories that show what traps to avoid. - Presents the wisdom of community experts, derived from a summit on software analytics - Provides contributed chapters that share discrete ideas and technique from the trenches - Covers top areas of concern, including mining security and social data, data visualization, and cloud-based data - Presented in clear chapters designed to be applicable across many domains

Data Analytics for Intelligent Transportation Systems provides in-depth coverage of data-enabled methods for analyzing intelligent transportation systems (ITS), including the tools needed to implement these methods using big data analytics and other computing techniques. The book examines the major characteristics of connected transportation systems, along with the fundamental concepts of how to analyze the data they produce. It explores collecting, archiving, processing, and distributing the data, designing data infrastructures, data management and delivery systems, and the required hardware and software technologies. It presents extensive coverage of existing and forthcoming intelligent transportation systems and data analytics technologies. All fundamentals/concepts presented in this book are explained in the context of ITS. Users will learn everything from the basics of different ITS data types and characteristics to how to evaluate alternative data analytics for different ITS applications. They will discover how to design effective data visualizations, tactics on the planning process, and how to evaluate alternative data analytics for different connected transportation applications, along with key safety and environmental applications for both commercial and passenger vehicles, data privacy and security issues, and the role of social media data in traffic planning. Data Analytics for Intelligent Transportation Systems will prepare an educated ITS workforce and tool builders to make the vision for safe, reliable, and environmentally sustainable intelligent transportation systems a reality. It serves as a primary or supplemental textbook for upper-level undergraduate and graduate ITS courses and a valuable reference for ITS practitioners. - Utilizes real ITS examples to facilitate a quicker grasp of materials presented - Contains contributors from both leading academic and commercial domains - Explains how to design effective data visualizations, tactics on the planning process, and how to evaluate alternative data analytics for different connected transportation applications - Includes exercise problems in each chapter to help readers apply and master the learned fundamentals, concepts, and techniques - New to the second edition: Two new chapters on Quantum Computing in Data Analytics and Society and Environment in ITS Data Analytics

An up-to-date guide for using massive amounts of data and novel technologies to design, build, and maintain better systems engineering Systems Engineering in the Fourth Industrial Revolution: Big Data, Novel Technologies, and Modern Systems Engineering offers a guide to the recent changes in systems engineering prompted by the current challenging and innovative industrial environment called the Fourth Industrial Revolution—INDUSTRY 4.0. This book contains advanced models, innovative practices, and state-of-the-art research findings on systems engineering. The contributors, an international panel of experts on the topic, explore the key elements in systems engineering that have shifted towards data collection and analytics, available and used in the design and development of systems and also in the later life-cycle stages of use and retirement. The contributors address the issues in a system in which the system involves data in its operation, contrasting with earlier approaches in which data, models, and algorithms were less involved in the function of the system. The book covers a wide range of topics including five systems engineering domains: systems engineering and systems thinking; systems software and process engineering; the digital factory; reliability and maintainability modeling and analytics; and organizational aspects of systems engineering. This important resource: Presents new and advanced approaches, methodologies, and tools for designing, testing, deploying, and maintaining advanced complex systems Explores effective evidence-based risk management practices Describes an integrated approach to safety, reliability, and cyber security based on system theory Discusses entrepreneurship as a multidisciplinary system Emphasizes technical merits of systems engineering concepts by providing technical models Written for systems engineers, Systems Engineering in the Fourth Industrial Revolution offers an up-to-date resource that contains the best practices and most recent research on the topic of systems engineering.

A textbook covering data-science and machine learning methods for modelling and control in engineering and science, with Python and MATLAB®.

Experts in data analytics and power engineering present techniques addressing the needs of modern power systems, covering theory and applications related to power system reliability, efficiency, and security. With topics spanning large-scale and distributed optimization, statistical learning, big data analytics, graph theory, and game theory, this is an essential resource for graduate students and researchers in academia and industry with backgrounds in power systems engineering, applied mathematics, and computer science.

This book provides a step-by-step guidance on how to implement analytical methods in project risk management. The text focuses on engineering design and construction projects and as such is suitable for graduate students in engineering, construction, or project management, as well as practitioners aiming to develop, improve, and/or simplify corporate project management processes. The book places emphasis on building data-driven models for additive-incremental risks, where data can be collected on project sites, assembled from queries of corporate databases, and/or generated using procedures for eliciting experts’ judgments. While the presented models are mathematically inspired, they are nothing beyond what an engineering graduate is expected to know: some algebra, a little calculus, a little statistics, and, especially, undergraduate-level understanding of the probability theory. The book is organized in three parts and fourteen chapters. In Part I the authors provide the general introduction to risk and uncertainty analysis applied to engineering construction projects. The basic formulations and the methods for risk assessment used during project planning phase are discussed in Part II, while in Part III the authors present the methods for monitoring and (re)assessment of risks during project execution.

Data Analytics in Biomedical Engineering and Healthcare explores key applications using data analytics, machine learning, and deep learning in health sciences and biomedical data. The book is useful for those working with big data analytics in biomedical research, medical industries, and medical research scientists. The book covers health analytics, data science, and machine and deep learning applications for biomedical data, covering areas such as predictive health analysis, electronic health records, medical image analysis, computational drug discovery, and genome structure prediction using predictive modeling. Case studies demonstrate big data applications in healthcare using the MapReduce and Hadoop frameworks. - Examines the development and application of data analytics applications in biomedical data - Presents innovative classification and regression models for predicting various diseases - Discusses genome structure prediction using predictive modeling - Shows readers how to develop clinical decision support systems - Shows researchers and specialists how to use hybrid learning for better medical diagnosis, including case studies of healthcare applications using the MapReduce and Hadoop frameworks

Build a data platform to the industry-leading standards set by Microsoft’s own infrastructure. Summary In Data Engineering on Azure you will learn how to: Pick the right Azure services for different data scenarios Manage data inventory Implement production quality data modeling, analytics, and machine learning workloads Handle data governance Using DevOps to increase reliability Ingesting, storing, and distributing data Apply best practices for compliance and access control Data Engineering on Azure reveals the data management patterns and techniques that support Microsoft’s own massive data infrastructure. Author Vlad Riscutia, a data engineer at Microsoft, teaches you to bring an engineering rigor to your data platform and ensure that your data prototypes function just as well under the pressures of production. You'll implement common data modeling patterns, stand up cloud-native data platforms on Azure, and get to grips with DevOps for both analytics and machine learning. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology Build secure, stable data platforms that can scale to loads of any size. When a project moves from the lab into production, you need confidence that it can stand up to real-world challenges. This book teaches you to design and implement cloud-based data infrastructure that you can easily monitor, scale, and modify. About the book In Data Engineering on Azure you’ll learn the skills you need to build and maintain big data platforms in massive enterprises. This invaluable guide includes clear, practical guidance for setting up infrastructure, orchestration, workloads, and governance. As you go, you’ll set up efficient machine learning pipelines, and then master time-saving automation and DevOps solutions. The Azure-based examples are easy to reproduce on other cloud platforms. What's inside Data inventory and data governance Assure data quality, compliance, and distribution Build automated pipelines to increase reliability Ingest, store, and distribute data Production-quality data modeling, analytics, and machine learning About the reader For data engineers familiar with cloud computing and DevOps. About the author Vlad Riscutia is a software architect at Microsoft. Table of Contents 1 Introduction PART 1 INFRASTRUCTURE 2 Storage 3 DevOps 4 Orchestration PART 2 WORKLOADS 5 Processing 6 Analytics 7 Machine learning PART 3 GOVERNANCE 8 Metadata 9 Data quality 10 Compliance 11 Distributing data

Feature engineering plays a vital role in big data analytics. Machine learning and data mining algorithms cannot work without data. Little can be achieved if there are few features to represent the underlying data objects, and the quality of results of those algorithms largely depends on the quality of the available features. Feature Engineering for Machine Learning and Data Analytics provides a comprehensive introduction to feature engineering, including feature generation, feature extraction, feature transformation, feature selection, and feature analysis and evaluation. The book presents key concepts, methods, examples, and applications, as well as chapters on feature engineering for major data types such as texts, images, sequences, time series, graphs, streaming data, software engineering data, Twitter data, and social media data. It also contains generic feature generation approaches, as well as methods for generating tried-and-tested, hand-crafted, domain-specific features. The first chapter defines the concepts of features and feature engineering, offers an overview of the book, and provides pointers to topics not covered in this book. The next six chapters are devoted to feature engineering, including feature generation for specific data types. The subsequent four chapters cover generic approaches for feature engineering, namely feature selection, feature transformation based feature engineering, deep learning based feature engineering, and pattern based feature generation and engineering. The last three chapters discuss feature engineering for social bot detection, software management, and Twitter-based applications respectively. This book can be used as a reference for data analysts, big data scientists, data preprocessing workers, project managers, project developers, prediction modelers, professors, researchers, graduate students, and upper level undergraduate students. It can also be used as the primary text for courses on feature engineering, or as a supplement for courses on machine learning, data mining, and big data analytics.