Human driving decisions are the leading cause of road fatalities. Autonomous driving naturally eliminates such incompetent decisions and thus can improve traffic safety and efficiency. Deep reinforcement learning (DRL) has shown great potential in learning complex tasks. Recently, researchers investigated various DRL-based approaches for autonomous driving. However, exploiting multi-modal fusion to generate pixel-wise perception and motion prediction and then leveraging these predictions to train a latent DRL has not been targeted yet. Unlike other DRL algorithms, the latent DRL algorithm distinguishes representation learning from task learning, enhancing sampling efficiency for reinforcement learning. In addition, supplying the latent DRL algorithm with accurate perception and motion prediction simplifies the surrounding urban scenes, improving training and thus learning a better driving policy. To that end, this Ph.D. research initially develops LiCaNext, a novel real-time multi-modal fusion network to produce accurate joint perception and motion prediction at a pixel level. Our proposed approach relies merely on a LIDAR sensor, where its multi-modal input is composed of bird's-eye view (BEV), range view (RV), and range residual images. Further, this Ph.D. thesis proposes leveraging these predictions with another simple BEV image to train a sequential latent maximum entropy reinforcement learning (MaxEnt RL) algorithm. A sequential latent model is deployed to learn a more compact latent representation from high-dimensional inputs. Subsequently, the MaxEnt RL model trains on this latent space to learn a driving policy. The proposed LiCaNext is trained on the public nuScenes dataset. Results demonstrated that LiCaNext operates in real-time and performs better than the state-of-the-art in perception and motion prediction, especially for small and distant objects. Furthermore, simulation experiments are conducted on CARLA to evaluate the performance of our proposed approach that exploits LiCaNext predictions to train sequential latent MaxEnt RL algorithm. The simulated experiments manifest that our proposed approach learns a better driving policy outperforming other prevalent DRL-based algorithms. The learned driving policy achieves the objectives of safety, efficiency, and comfort. Experiments also reveal that the learned policy maintains its effectiveness under different environments and varying weather conditions.

The nine-volume set constitutes the refereed proceedings of the 30th International Conference on Neural Information Processing, ICONIP 2023, held in Changsha, China, in November 2023. The 1274 papers presented in the proceedings set were carefully reviewed and selected from 652 submissions. The ICONIP conference aims to provide a leading international forum for researchers, scientists, and industry professionals who are working in neuroscience, neural networks, deep learning, and related fields to share their new ideas, progress, and achievements.

Self-driving vehicles are a rapidly growing area of research and expertise. Theories and Practice of Self-Driving Vehicles presents a comprehensive introduction to the technology of self driving vehicles across the three domains of perception, planning and control. The title systematically introduces vehicle systems from principles to practice, including basic knowledge of ROS programming, machine and deep learning, as well as basic modules such as environmental perception and sensor fusion. The book introduces advanced control algorithms as well as important areas of new research. This title offers engineers, technicians and students an accessible handbook to the entire stack of technology in a self-driving vehicle. Theories and Practice of Self-Driving Vehicles presents an introduction to self-driving vehicle technology from principles to practice. Ten chapters cover the full stack of driverless technology for a self-driving vehicle. Written by two authors experienced in both industry and research, this book offers an accessible and systematic introduction to self-driving vehicle technology. - Provides a comprehensive introduction to the technology stack of a self-driving vehicle - Covers the three domains of perception, planning and control - Offers foundational theory and best practices - Introduces advanced control algorithms and high-potential areas of new research - Gives engineers, technicians and students an accessible handbook to self-driving vehicle technology and applications

To drive a vehicle fully autonomously, an intelligent system needs to be capable of having accurate perception and comprehensive understanding of the surroundings, making reasonable predictions of the progressing of the scenario, and executing safe, comfortable, as well as efficient control actions. Currently, these requirements are mostly fulfilled by the intelligence of human drivers. During past decades, with the development of machine learning and computer science, artificial intelligence starts to show better-than-human performance on more and more practical applications, while autonomous driving is still one of the most attractive and difficult unconquered challenges. This thesis studies the challenges of autonomous driving on its safety and interaction with the surrounding vehicles, and how deep reinforcement learning methods could help address these challenges. Reinforcement learning (RL) is an important paradigm of machine learning which focuses on learning sequential decision-making policies which interact with the task environment. Combining with deep neural networks, the recent development of deep reinforcement learning has shown promising results on control and decision-making tasks with high dimensional observations and complex strategies. The capability and achievements of deep reinforcement learning indicate a wide range of potential applications in autonomous driving. Focusing on autonomous driving safety and interactivity, this thesis presents novel contributions on topics including safe and robust reinforcement learning, reinforcement learning-based safety test, human driver modeling, and multi-agent reinforcement learning. This thesis begins with the study of deep reinforcement learning methods on autonomous driving safety, which is the most critical concern for all autonomous driving systems. We study the autonomous driving safety problem from two points of view: the first is the risk caused by the reinforcement learning control policies due to the mismatch between simulations and the real world; the second is the deep reinforcement learning-based safety test. In both problems, we explore the usage of adversary reinforcement learning agents on finding failures of the system with different focuses: on the first problem, the RL adversary is trained and applied at the learning stage of the control policy to guide it to learn more robust behaviors; on the second problem, the RL adversary is used at the test stage to find the most likely failures in the system. Different learning approaches are proposed and studied for the two problems. Another fundamental challenge for autonomous driving is the interaction between the autonomous vehicle and its surrounding vehicles, which requires accurate modeling of the behavior of surrounding drivers. In the second and third parts of the thesis, we study the surrounding driver modeling problem on three different levels: the action distribution level, the latent state level, and the reasoning level. On the action distribution level, we explore advanced policy representations for modeling the complex distribution of driver's control actions. On the latent state level, we study how to efficiently infer the latent states of surrounding drivers like their driving characteristics and intentions, and how it could be combined with the learning of autonomous driving decision-making policies. On the reasoning level, we investigate the reasoning process between multiple interacting agents and use this to build their behavior models through multi-agent reinforcement learning.

The next generation of autonomous vehicles will provide major improvements in traffic flow, fuel efficiency, and vehicle safety. Several challenges currently prevent the deployment of autonomous vehicles, one aspect of which is robust and adaptable vehicle control. Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalizing previously learned rules to new scenarios. For these reasons, the use of deep neural networks for vehicle control has gained significant interest. In this book, we introduce relevant deep learning techniques, discuss recent algorithms applied to autonomous vehicle control, identify strengths and limitations of available methods, discuss research challenges in the field, and provide insights into the future trends in this rapidly evolving field.

Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. It's representative of complex reinforcement learning tasks humans encounter in real life. The aim of this thesis is to explore the effectiveness of multi-objective reinforcement learning for such tasks characterized by autonomous driving. In particular, it shows that: 1. Multi-objective reinforcement learning is effective at overcoming some of the difficulties faced by scalar-reward reinforcement learning, and a multi-objective DQN agent based on a variant of thresholded lexicographic Q-learning is successfully trained to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules. 2. Data efficiency of (multi-objective) reinforcement learning can be significantly improved by exploiting the factored structure of a task. Specifically, factored Q functions learned on the factored state space can be used as features to the original Q function to speed up learning. 3. Inclusion of history-dependent policies enables an intuitive exact algorithm for multi-objective reinforcement learning with thresholded lexicographic order.

Multimodal Perception and Secure State Estimation for Robotic Mobility Platforms Enables readers to understand important new trends in multimodal perception for mobile robotics This book provides a novel perspective on secure state estimation and multimodal perception for robotic mobility platforms such as autonomous vehicles. It thoroughly evaluates filter-based secure dynamic pose estimation approaches for autonomous vehicles over multiple attack signals and shows that they outperform conventional Kalman filtered results. As a modern learning resource, it contains extensive simulative and experimental results that have been successfully implemented on various models and real platforms. To aid in reader comprehension, detailed and illustrative examples on algorithm implementation and performance evaluation are also presented. Written by four qualified authors in the field, sample topics covered in the book include: Secure state estimation that focuses on system robustness under cyber-attacks Multi-sensor fusion that helps improve system performance based on the complementary characteristics of different sensors A geometric pose estimation framework to incorporate measurements and constraints into a unified fusion scheme, which has been validated using public and self-collected data How to achieve real-time road-constrained and heading-assisted pose estimation This book will appeal to graduate-level students and professionals in the fields of ground vehicle pose estimation and perception who are looking for modern and updated insight into key concepts related to the field of robotic mobility platforms.

Multimodal Scene Understanding: Algorithms, Applications and Deep Learning presents recent advances in multi-modal computing, with a focus on computer vision and photogrammetry. It provides the latest algorithms and applications that involve combining multiple sources of information and describes the role and approaches of multi-sensory data and multi-modal deep learning. The book is ideal for researchers from the fields of computer vision, remote sensing, robotics, and photogrammetry, thus helping foster interdisciplinary interaction and collaboration between these realms. Researchers collecting and analyzing multi-sensory data collections – for example, KITTI benchmark (stereo+laser) - from different platforms, such as autonomous vehicles, surveillance cameras, UAVs, planes and satellites will find this book to be very useful. - Contains state-of-the-art developments on multi-modal computing - Shines a focus on algorithms and applications - Presents novel deep learning topics on multi-sensor fusion and multi-modal deep learning

Building self-driving vehicles is exciting and promising. It is going to transform the way we live and provide safety, efficiency, and mobility for everyone. In this thesis, I present a collection of our work in the direction of building smart self-driving vehicles from understanding convolutional neural networks (CNNs) to the full autonomy stacks, including perception, prediction, and planning. First, we study the property of CNNs and provide theoretical analysis on the receptive field. We use Fourier transform and center limit theorem to show that the effective receptive field only occupies a fraction of the theoretical one, especially in deep models. This can provide insights for future CNNs design. Next, we push the state of the art for tasks in the autonomy stacks using deep CNNs. For depth estimation with stereo cameras, we develop a deep matching network that simplifies previous approaches by using a dot-product layer and incorporates a multi-class classification loss, allowing the network to calibrate scores among larger contexts. These greatly improve the runtime and achieve much better matching results. Furthermore, we extend the idea to optical flow where the matching is done in 2D space. Combined with an instance segmentation algorithm, we achieve state-of-the-art results on KITTI optical flow benchmark. Third, we propose new formulations for the self-driving system, i.e. using a joint model for 3D detection, prediction, and tracking. Our model takes a sequence of consecutive LiDAR sweeps and predicts the bounding boxes for vehicles at the current time step as well as one second into the future. Tracking is done by greedily checking the overlap between current detections and predictions from the past. This approach bridges the gap between perception and prediction to achieve better performance both in accuracy and runtime. Then, we take this one step further to construct a deep structured model for interpretable neural motion planning, where the CNN also predicts a 3D cost volume, expanding on time and x-y spatial dimensions. The final planning trajectory is generated with a sampling-based inference algorithm. We conduct offline testing and show that our approach is better than several baselines.