This book treats visual feedback control of mechanical systems, mostly robot manipulators. It not only deals with image processing techniques and robot control schemes but also covers the latest investigation of the design of the visual servo mechanism based on modern linear and nonlinear control theory, the adaptive control scheme, fuzzy logic, and neural networks. New concepts for utilizing visual sensory information for real-time manipulator control are derived and the performances are evaluated through simulations and/or experiments.The contributors to this book are robotics specialists from all over the world. The book gives a practical perspective on visual servoing to researchers, engineers, and students working in this area.

The goal of this book is to introduce the visional application by excellent researchers in the world currently and offer the knowledge that can also be applied to another field widely. This book collects the main studies about machine vision currently in the world, and has a powerful persuasion in the applications employed in the machine vision. The contents, which demonstrate that the machine vision theory, are realized in different field. For the beginner, it is easy to understand the development in the vision servoing. For engineer, professor and researcher, they can study and learn the chapters, and then employ another application method.

The author has maintained two open-source MATLAB Toolboxes for more than 10 years: one for robotics and one for vision. The key strength of the Toolboxes provide a set of tools that allow the user to work with real problems, not trivial examples. For the student the book makes the algorithms accessible, the Toolbox code can be read to gain understanding, and the examples illustrate how it can be used —instant gratification in just a couple of lines of MATLAB code. The code can also be the starting point for new work, for researchers or students, by writing programs based on Toolbox functions, or modifying the Toolbox code itself. The purpose of this book is to expand on the tutorial material provided with the toolboxes, add many more examples, and to weave this into a narrative that covers robotics and computer vision separately and together. The author shows how complex problems can be decomposed and solved using just a few simple lines of code, and hopefully to inspire up and coming researchers. The topics covered are guided by the real problems observed over many years as a practitioner of both robotics and computer vision. It is written in a light but informative style, it is easy to read and absorb, and includes a lot of Matlab examples and figures. The book is a real walk through the fundamentals of robot kinematics, dynamics and joint level control, then camera models, image processing, feature extraction and epipolar geometry, and bring it all together in a visual servo system. Additional material is provided at http://www.petercorke.com/RVC

Visual servoing is a well-known approach to guide robots using visual information. Image processing, robotics, and control theory are combined in order to control the motion of a robot depending on the visual information extracted from the images captured by one or several cameras. With respect to vision issues, a number of issues are currently being addressed by ongoing research, such as the use of different types of image features (or different types of cameras such as RGBD cameras), image processing at high velocity, and convergence properties. As shown in this book, the use of new control schemes allows the system to behave more robustly, efficiently, or compliantly, with fewer delays. Related issues such as optimal and robust approaches, direct control, path tracking, or sensor fusion are also addressed. Additionally, we can currently find visual servoing systems being applied in a number of different domains. This book considers various aspects of visual servoing systems, such as the design of new strategies for their application to parallel robots, mobile manipulators, teleoperation, and the application of this type of control system in new areas.

Robots able to imitate human beings have been at the core of stories of science?ctionaswellasdreamsofinventorsforalongtime.Amongthe various skills that Mother Nature has provided us with and that often go forgotten, the ability of sight is certainly one of the most important. Perhaps inspired by tales of Isaac Asimov, comics and cartoons, and surely helped by the progress of electronics in recent decades, researchers have progressively made the dream of creating robots able to move and operate by exploiting arti?cial vision a concrete reality. Technically speaking, we would say that these robots position themselves and their end-e?ectors by using the view provided by some arti?cial eyes as feedback information. Indeed, the arti?cial eyes are visual sensors such as cameras that have the function to acquire an image of the environment. Such an image describes if and how the robot is moving toward the goal and hence constitutes feedback information. This procedure is known in robotics with the term visual servoing, and it is nothing else than an imitation of the intrinsic mechanism that allows human beings to realize daily tasks such as reaching the door of the house or grasping a cup of co?ee.

Sight and touch are two elementary, but highly complementary senses - for humans as well as for robots. This monograph develops an integrated vision/force control approach for robotics, combining the advantages of both types of sensors while overcoming their individual drawbacks. It shows how integrated vision/force control improves the task quality in the sense of increased accuracy and execution velocity and widens the range of feasible tasks. The unique feature of this work lies in its comprehensive treatment of the problem from the theoretical development of the various schemes down to the real-time implementation of interaction control algorithms on an industrial robot. The presented approach and its potential impact on the performance of the next generation of robots is starting to be recognized by major manufacturers worldwide.

Taking human factors into account, a visual servoing approach aims to facilitate robots with real-time situational information to accomplish tasks in direct and proximate collaboration with people. A hybrid visual servoing algorithm, a combination of the classical position-based and image-based visual servoing, is applied to the whole task space. A model-based tracker monitors the human activities, via matching the human skeleton representation and the image of people in image. Grasping algorithms are implemented to compute grasp points based on the geometrical model of the robot gripper. Whilst major challenges of human-robot interactive object transfer are visual occlusions and making grasping plans, this work proposes a new method of visually guiding a robot with the presence of partial visual occlusion, and elaborate the solution to adaptive robotic grasping.

Vision-based control of wheeled mobile robots is an interesting field of research from a scientific and even social point of view due to its potential applicability. This book presents a formal treatment of some aspects of control theory applied to the problem of vision-based pose regulation of wheeled mobile robots. In this problem, the robot has to reach a desired position and orientation, which are specified by a target image. It is faced in such a way that vision and control are unified to achieve stability of the closed loop, a large region of convergence, without local minima and good robustness against parametric uncertainty. Three different control schemes that rely on monocular vision as unique sensor are presented and evaluated experimentally. A common benefit of these approaches is that they are valid for imaging systems obeying approximately a central projection model, e.g., conventional cameras, catadioptric systems and some fisheye cameras. Thus, the presented control schemes are generic approaches. A minimum set of visual measurements, integrated in adequate task functions, are taken from a geometric constraint imposed between corresponding image features. Particularly, the epipolar geometry and the trifocal tensor are exploited since they can be used for generic scenes. A detailed experimental evaluation is presented for each control scheme.

This monograph covers theoretical and practical aspects of the problem of autonomous guiding of unmanned aerial manipulators using visual information. For the estimation of the vehicle state (position, orientation, velocity, and acceleration), the authors propose a method that relies exclusively on the use of low-cost and highrate sensors together with low-complexity algorithms. This is particularly interesting for applications in which on board computation with low computation power is needed. Another relevant topic covered in this monograph is visual servoing. The authors present an uncalibrated visual servo scheme, capable of estimating at run time, the camera focal length from the observation of a tracked target. The monograph also covers several control techniques, which achieve a number of tasks, such as robot and arm positioning, improve stability and enhance robot arm motions. All methods discussed in this monograph are demonstrated in simulation and through real robot experimentation. The text is appropriate for readers interested in state estimation and control of aerial manipulators, and is a reference book for people who work in mobile robotics research in general.