Deep learning is widely applied to sparse 3D data to perform challenging tasks, e.g., 3D object detection and semantic segmentation. However, the high performance of deep learning comes with high costs, including computational costs and the effort to capture and label data. This work investigates and improves the efficiency of deep learning for sparse 3D data to overcome the obstacles to the further development of this technology.

Deep Learning through Sparse Representation and Low-Rank Modeling bridges classical sparse and low rank models—those that emphasize problem-specific Interpretability—with recent deep network models that have enabled a larger learning capacity and better utilization of Big Data. It shows how the toolkit of deep learning is closely tied with the sparse/low rank methods and algorithms, providing a rich variety of theoretical and analytic tools to guide the design and interpretation of deep learning models. The development of the theory and models is supported by a wide variety of applications in computer vision, machine learning, signal processing, and data mining. This book will be highly useful for researchers, graduate students and practitioners working in the fields of computer vision, machine learning, signal processing, optimization and statistics. - Combines classical sparse and low-rank models and algorithms with the latest advances in deep learning networks - Shows how the structure and algorithms of sparse and low-rank methods improves the performance and interpretability of Deep Learning models - Provides tactics on how to build and apply customized deep learning models for various applications

Using massive computation, deep learning allows machines to translate large amounts of data into models that accurately predict the real world, enabling powerful applications like virtual assistants and autonomous vehicles. As datasets and computer systems have continued to grow in scale, so has the quality of machine learning models, creating an expensive appetite in practitioners and researchers for data and computation. To address this demand, this dissertation discusses ways to measure and improve both the computational and data efficiency of deep learning. First, we introduce DAWNBench and MLPerf as a systematic way to measure end-to-end machine learning system performance. Researchers have proposed numerous hardware, software, and algorithmic optimizations to improve the computational efficiency of deep learning. While some of these optimizations perform the same operations faster (e.g., increasing GPU clock speed), many others modify the semantics of the training procedure (e.g., reduced precision) and can even impact the final model's accuracy on unseen data. Because of these trade-offs between accuracy and computational efficiency, it has been difficult to compare and understand the impact of these optimizations. We propose and evaluate a new metric, time-to-accuracy, that can be used to compare different system designs and use it to evaluate high performing systems by organizing two public benchmark competitions, DAWNBench and MLPerf. MLPerf has now grown into an industry standard benchmark co-organized by over 70 organizations. Second, we present ways to perform data selection on large-scale datasets efficiently. Data selection methods, such as active learning and core-set selection, improve the data efficiency of machine learning by identifying the most informative data points to label or train on. Across the data selection literature, there are many ways to identify these training examples. However, classical data selection methods are prohibitively expensive to apply in deep learning because of the larger datasets and models. To make these methods tractable, we propose (1) "selection via proxy" (SVP) to avoid expensive training and reduce the computation per example and (2) "similarity search for efficient active learning and search" (SEALS) to reduce the number of examples processed. Both methods lead to order of magnitude performance improvements, making techniques like active learning on billions of unlabeled images practical for the first time.

This book comprises theoretical foundations to deep learning, machine learning and computing system, deep learning algorithms, and various deep learning applications. The book discusses significant issues relating to deep learning in data analytics. Further in-depth reading can be done from the detailed bibliography presented at the end of each chapter. Besides, this book's material includes concepts, algorithms, figures, graphs, and tables in guiding researchers through deep learning in data science and its applications for society. Deep learning approaches prevent loss of information and hence enhance the performance of data analysis and learning techniques. It brings up many research issues in the industry and research community to capture and access data effectively. The book provides the conceptual basis of deep learning required to achieve in-depth knowledge in computer and data science. It has been done to make the book more flexible and to stimulate further interest in topics. All these help researchers motivate towards learning and implementing the concepts in real-life applications.

This book presents deep learning techniques, concepts, and algorithms to classify and analyze big data. Further, it offers an introductory level understanding of the new programming languages and tools used to analyze big data in real-time, such as Hadoop, SPARK, and GRAPHX. Big data analytics using traditional techniques face various challenges, such as fast, accurate and efficient processing of big data in real-time. In addition, the Internet of Things is progressively increasing in various fields, like smart cities, smart homes, and e-health. As the enormous number of connected devices generate huge amounts of data every day, we need sophisticated algorithms to deal, organize, and classify this data in less processing time and space. Similarly, existing techniques and algorithms for deep learning in big data field have several advantages thanks to the two main branches of the deep learning, i.e. convolution and deep belief networks. This book offers insights into these techniques and applications based on these two types of deep learning. Further, it helps students, researchers, and newcomers understand big data analytics based on deep learning approaches. It also discusses various machine learning techniques in concatenation with the deep learning paradigm to support high-end data processing, data classifications, and real-time data processing issues. The classification and presentation are kept quite simple to help the readers and students grasp the basics concepts of various deep learning paradigms and frameworks. It mainly focuses on theory rather than the mathematical background of the deep learning concepts. The book consists of 5 chapters, beginning with an introductory explanation of big data and deep learning techniques, followed by integration of big data and deep learning techniques and lastly the future directions.

This book presents a detailed review of the state of the art in deep learning approaches for semantic object detection and segmentation in medical image computing, and large-scale radiology database mining. A particular focus is placed on the application of convolutional neural networks, with the theory supported by practical examples. Features: highlights how the use of deep neural networks can address new questions and protocols, as well as improve upon existing challenges in medical image computing; discusses the insightful research experience of Dr. Ronald M. Summers; presents a comprehensive review of the latest research and literature; describes a range of different methods that make use of deep learning for object or landmark detection tasks in 2D and 3D medical imaging; examines a varied selection of techniques for semantic segmentation using deep learning principles in medical imaging; introduces a novel approach to interleaved text and image deep mining on a large-scale radiology image database.

The phenomenal success of deep learning in the past decade has been mostly driven by the construction of increasingly large deep neural network models. These models usually impose an ideal assumption that there are sufficient resources, including large-scale parameters, sufficient data, and massive computation, for the optimization. However, this assumption usually fails in real-world scenarios. For example, computer memory may be limited as in edge devices, large-scale data are difficult to obtain due to expensive costs and privacy constraints, and computational power is constrained as in most university labs. As a result, these resource discrepancy issues have hindered the democratization of deep learning techniques in many AI applications, and the development of efficient deep learning methods that can adapt to different resource constraints is of great importance. In this dissertation, I will present my Ph.D. research concerned with the aforementioned resource discrepancy issues to free AI from the parameter-data-computation hungry beast in three threads. The first thread focuses on data efficiency in deep learning technologies. This thread extends advances in deep learning to scenarios with small, sensitive, or unlabeled data, accelerating the acceptance and adoption of AI in real-world applications. In particular, I study self-supervised learning to remove the dependency on labels, few-shot learning to free model from a large number of samples, and and attentive learning to take full advantage of heterogeneous information sources. The second thread of my work focuses on advances of parameter efficiency in deep learning technologies, which enable us to democratize powerful deep learning models at scale to bridge computer memory divide and improve the adaptability of models in dynamic environments. I study network sparsity, i.e., the technology to prune networks, and network modularity, i.e., the technology to modularize neural networks into multiple modules, each of which is a function with its own parameters. The third thread focuses on computation efficiency of deep learning models, from inference to training, reducing the energy consumption of models, promoting environmental sustainability, and complementing data efficiency. More specifically, I study task-agnostic model compression, the task of generating efficient compressed models without utilizing the downstream task label information, avoiding the repetitive compression process, which saves much training cost.

Recent advancements in imaging techniques and image analysis has broadened the horizons for their applications in various domains. Image analysis has become an influential technique in medical image analysis, optical character recognition, geology, remote sensing, and more. However, analysis of images under constrained and unconstrained environments require efficient representation of the data and complex models for accurate interpretation and classification of data. Deep learning methods, with their hierarchical/multilayered architecture, allow the systems to learn complex mathematical models to provide improved performance in the required task. The Handbook of Research on Deep Learning-Based Image Analysis Under Constrained and Unconstrained Environments provides a critical examination of the latest advancements, developments, methods, systems, futuristic approaches, and algorithms for image analysis and addresses its challenges. Highlighting concepts, methods, and tools including convolutional neural networks, edge enhancement, image segmentation, machine learning, and image processing, the book is an essential and comprehensive reference work for engineers, academicians, researchers, and students.

Deep Learning for Medical Image Analysis, Second Edition is a great learning resource for academic and industry researchers and graduate students taking courses on machine learning and deep learning for computer vision and medical image computing and analysis. Deep learning provides exciting solutions for medical image analysis problems and is a key method for future applications. This book gives a clear understanding of the principles and methods of neural network and deep learning concepts, showing how the algorithms that integrate deep learning as a core component are applied to medical image detection, segmentation, registration, and computer-aided analysis.· Covers common research problems in medical image analysis and their challenges · Describes the latest deep learning methods and the theories behind approaches for medical image analysis · Teaches how algorithms are applied to a broad range of application areas including cardiac, neural and functional, colonoscopy, OCTA applications and model assessment · Includes a Foreword written by Nicholas Ayache