多主体强化学习协作策略研究

内容简介

多主体的研究与应用是近年来备受关注的热点领 域，多主体强化学习理论与方法、多主体协作策略的 研究是该领域重要研究方向，其理论和应用价值极为 广泛，备受广大从事计算机应用、人工智能、自动控 制、以及经济管理等领域研究者的关注。孙若莹、赵 刚所著的《多主体强化学习协作策略研究》清晰地介 绍了多主体、强化学习及多主体协作等基本概念和基 础内容，明确地阐述了有关多主体强化学习、协作策 略研究的发展过程及动向，深入地探讨了多主体 强化学习与协作策略的理论与方法，具体地分析了多 主体强化学习与协作策略在相关研究领域的应用方法 。
     全书系统脉络清晰、基本概念清楚、图表分析直 观，注重内容的体系化和实用性。通过本书的阅读和 学习，读者即可掌握多主体强化学习及协作策略的理 论和方法，更可了解在实际工作中应用这些研究成果 的手段。本书可作为从事计算机应用、人工智能、自 动控制、以及经济管理等领域研究者的学习和阅读参 考，同时高等院校相关专业研究生以及人工智能爱好 者也可从中获得借鉴。

目录

Chapter 1 Introduction
1.1 Reinforcement Learning
1.1.1 Generality of Reinforcement Learning
1.1.2 Reinforcement Learning on Markov Decision Processes
1.1.3 Integrating Reinforcement Learning into Agent Architecture
1.2 Multiagent Reinforcement Learning
1.2.1 Multiagent Systems
1.2.2 Reinforcement Learning in Multiagent Systems
1.2.3 Learning and Coordination in Multiagent Systems
1.3 Ant System for Stochastic Combinatorial Optimization
1.3.1 Ants Forage Behavior
1.3.2 Ant colony Optimization
1.3.3 MAX-MIN Ant System
1.4 Motivations and Consequences
1.5 Book Summary
Bibliography
Chapter 2 Reinforcement Learning and Its Combination with Ant colony System
2.1 Introduction
2.2 Investigation into Reinforcement Learning and Swarm Intelligence
2.2.1 Temporal Differences Learning Method
2.2.2 Active Exploration and Experience Replay in Reinforcement Learning
2.2.3 Ant colony System for Traveling Salesman Problem
2.3 The Q-ACS Multiagent Learning Method
2.3.1 The Q-ACS Learning Algorithm
2.3.2 Some Properties of the Q-ACS Learning Method
2.3.3 Relation with Ant-Q Learning Method
2.4 Simulations and Results
2.5 Conclusions
Bibliography
Chapter 3 Multiagent Learning Methods based on Indirect Media Information Sharing
3.1 Introduction
3.2 The Multiagent Learning Method Considering Statistics Features
3.2.1 Accelerated K-certainty Exploration
3.2.2 The T-ACS Learning Algorithm
3.3 The Heterogeneous Agents Learning
3.3.1 The D-ACS Learning Algorithm
3.3.2 Some Discussions about the D-ACS Learning Algorithm
3.4 Comparisons with Related State-of-the-arts
3.5 Simulations and Results
3.5.1 Experimental Results on Hunter Game
3.5.2 Experimental Results on Traveling Salesman Problem
3.6 Conclusions
Bibliography
Chapter 4 Action Conversion Mechanism in Multiagent Reinforcement Learning
4.1 Introduction
4.2 Model-based Reinforcement Learning
4.2.1 Dyna-Q Architecture
4.2.2 Prioritized Sweeping Method
4.2.3 Minimax Search and Reinforcement Learning
4.2.4 RTP-Q Learning
4.3 The Q-ac Multiagent Reinforcement Learning
4.3.1 Task Model
4.3.2 Converting Action
4.3.3 Multiagent Cooperation Methods
4.3.4 Q-value Update
4.3.5 The Q-ac Learning Algorithm
4.3.6 Using Adversarial Action Instead o{ ~ Probability Exploration
4.4 Simulations and Results
4.5 Conclusions
Bibliography
Chapter 5 Multiagent Learning Approaches Applied to Vehicle Routing Problems
5.1 Introduction
5.2 Related State-of-the-arts
5.2.1 Some Heuristic Algorithms
5.2.2 The Vehicle Routing Problem with Time Windows
5.3 The Mu