The production control of flexible manufacturing systems is a relevant component that must go along with the requirements of being flexible in terms of new product variants, new machine skills and reaction to unforeseen events during runtime. This work focuses on developing a reactive job-shop scheduling system for flexible and re-configurable manufacturing systems. Reinforcement Learning approaches are therefore investigated for the concept of multiple agents that control products including transportation and resource allocation. About the author Schirin Bär researched at the RWTH-Aachen University at the Institute for Information Management in Mechanical Engineering (IMA) on the optimization of production control of flexible manufacturing systems using reinforcement learning. As operations manager and previously as an engineer, she developed and evaluated the research results based on real systems.

The production control of flexible manufacturing systems is a relevant component that must go along with the requirements of being flexible in terms of new product variants, new machine skills and reaction to unforeseen events during runtime. This work focuses on developing a reactive job-shop scheduling system for flexible and re-configurable manufacturing systems. Reinforcement Learning approaches are therefore investigated for the concept of multiple agents that control products including transportation and resource allocation.

This volume constitutes the refereed proceedings of the Third International Conference on Optimization and Learning, OLA 2020, held in Cádiz, Spain, in February 2020. The 23 full papers were carefully reviewed and selected from 55 submissions. The papers presented in the volume focus on the future challenges of optimization and learning methods, identifying and exploiting their synergies,and analyzing their applications in different fields, such as health, industry 4.0, games, logistics, etc.

This book constitutes the refereed proceedings of the 6th International Workshop on Artificial Intelligence and Pattern Recognition, IWAIPR 2018, held in Havana, Cuba, in September 2018. The 42 full papers presented were carefully reviewed and selected from 101 submissions. The papers promote and disseminate ongoing research on mathematical methods and computing techniques for artificial intelligence and pattern recognition, in particular in bioinformatics, cognitive and humanoid vision, computer vision, image analysis and intelligent data analysis, as well as their application in a number of diverse areas such as industry, health, robotics, data mining, opinion mining and sentiment analysis, telecommunications, document analysis, and natural language processing and recognition.

Distributed manufacturing has been an important trend in the industrial field, in which the production cost can be reduced through the cooperation among factories. In the real production, the random job arrivals are regular for the enterprises with daily delivered production tasks. In the paper, Distributed Job-shop Scheduling Problem (DJSP) with random job arrivals is studied. The distributed characteristics and the uncertain disturbance raise higher demands on the responsiveness and the self-adaptiveness of the scheduling method. To meet the scheduling requirements, a hierarchical Deep Reinforcement Learning (DRL) based multi-agent method Agentin is presented where the assigning agent (Agenta) and the sequencing agent (Agents) are respectively designed for job allocation and job sequencing, and they share the system information and extract the features they need independently. Agenta and Agents are both based on the specially-designed DQN framework, which has a variable threshold probability in the training stage, and it can balance the exploitation and exploration in the model training. For Agenta and Agents, two Markov Decision Process (MDP) formulations are established with elaborately-explored state features, rules-based action spaces and objective-oriented reward functions. Based on 1350 different production instances, the independent utility tests prove the effectiveness of the independent agents and the importance of the cooperation among the agents. The comparison test with the related algorithms validates the effectiveness of the integrated multi-agent method.

In a distributed system, a number of individually acting agents coexist. In order to achieve a common goal, coordinated cooperation between the agents is crucial. Many real-world applications are well-suited to be formulated in terms of spatially or functionally distributed entities. Job-shop scheduling represents one such application. Multi-agent reinforcement learning (RL) methods allow for automatically acquiring cooperative policies based solely on a specification of the desired joint behavior of the whole system. However, the decentralization of the control and observation of the system among independent agents has a significant impact on problem complexity. The author Thomas Gabel addresses the intricacy of learning and acting in multi-agent systems by two complementary approaches. He identifies a subclass of general decentralized decision-making problems that features provably reduced complexity. Moreover, he presents various novel model-free multi-agent RL algorithms that are capable of quickly obtaining approximate solutions in the vicinity of the optimum. All algorithms proposed are evaluated in the scope of various established scheduling benchmark problems.

Discover the latest developments in multi-robot coordination techniques with this insightful and original resource Multi-Agent Coordination: A Reinforcement Learning Approach delivers a comprehensive, insightful, and unique treatment of the development of multi-robot coordination algorithms with minimal computational burden and reduced storage requirements when compared to traditional algorithms. The accomplished academics, engineers, and authors provide readers with both a high-level introduction to, and overview of, multi-robot coordination, and in-depth analyses of learning-based planning algorithms. You'll learn about how to accelerate the exploration of the team-goal and alternative approaches to speeding up the convergence of TMAQL by identifying the preferred joint action for the team. The authors also propose novel approaches to consensus Q-learning that address the equilibrium selection problem and a new way of evaluating the threshold value for uniting empires without imposing any significant computation overhead. Finally, the book concludes with an examination of the likely direction of future research in this rapidly developing field. Readers will discover cutting-edge techniques for multi-agent coordination, including: An introduction to multi-agent coordination by reinforcement learning and evolutionary algorithms, including topics like the Nash equilibrium and correlated equilibrium Improving convergence speed of multi-agent Q-learning for cooperative task planning Consensus Q-learning for multi-agent cooperative planning The efficient computing of correlated equilibrium for cooperative q-learning based multi-agent planning A modified imperialist competitive algorithm for multi-agent stick-carrying applications Perfect for academics, engineers, and professionals who regularly work with multi-agent learning algorithms, Multi-Agent Coordination: A Reinforcement Learning Approach also belongs on the bookshelves of anyone with an advanced interest in machine learning and artificial intelligence as it applies to the field of cooperative or competitive robotics.

With the development and changes of industry and market demand, the personalized customization production mode with small batch and multiple batches has gradually become a new production mode. This makes production environment become more complex and dynamic. However, traditional production workshops cannot effectively adapt to this environment. Combing with new technologies, transforming traditional workshops into intelligent workshop to cope with new production mode become an urgent problem. Therefore, this paper proposes a multi-agent manufacturing system based on IoT for intelligent workshop. Meanwhile, this paper takes flexible job shop scheduling problem (FJSP) as a specific production scenario and establishes relevant mathematics model. To build the agent in intelligent workshop, this paper proposes a data-based with combination of virtual and physical agent (DB-VPA) which has information layer, software layer and physical layer. Then, based on the manufacturing system, this paper designs a dynamic scheduling mechanism for intelligent workshop. This method contains three aspects: (1) Modeling production process based on Markov decision process (MDP). (2) Designing communication mechanism for DB-VPAs. (3) Designing scheduling model combining with improve genetic programming and proximal policy optimization (IGP-PPO). Finally, relevant experiments are executed in a prototype experiment platform. The experiments indicate that the proposed method has superiority and generality in solving scheduling problem with dynamic events.

Both new rules are nonparametric and easy to be implemented in practice. A job release mechanism is applied to reduce job flowtimes (up to 20.3%) and work-in-process inventory (up to 33.1%), without worsening earliness and tardiness, and lead time performances. Flexible job shop scheduling problems are an important extension of the classical job shop scheduling problems and present additional complexity. A multi-agent scheduling method with job earliness and tardiness objectives in a flexible job shop environment is proposed. A new job routing and sequencing mechanism is developed. In this mechanism, different criteria for two kinds of jobs are proposed to route these jobs. Two sequencing algorithms based on existing methods are developed to deal with these two kinds of jobs. The proposed methodology is implemented in a flexible job shop environment. The computational results indicate that the proposed methodology is extremely fast.