Heterogeneous multi-agent search using reinforcement learning

Abstract

The dynamic and unpredictable nature of our world makes it difficult to design one autonomous robot that can efficiently adapt to all circumstances. Therefore, it makes sense to implement heterogeneous multi-robot systems to be able to solve complex tasks. The aim of this project is to search for targets in an unknown environment, using a team of heterogeneous agents/robots having different motion and sensing capabilities, employing reinforcement learning to distribute the agents efficiently and minimize searching time. The intuition behind heterogeneous search is considering different sensor capabilities, we want to find an online area decomposition to guide agents to search efficiently, finding how and where to go without many optimizations. A literature review of relevant work reveals that a majority of the current methods for multi-agent searching are either about homogeneous agents or using the same policy, under the same action space. There are very few papers describing heterogeneous multi-agent searching, and even those that do focus more on improving communication or other aspects. However, it is clear that heterogeneous multi-agent searching is an important emerging field and with the help of reinforcement learning, has the potential to lead to state-of-the-art performance on complicated tasks. For this project, we start with a model of ergodic search using homogeneous agents, then try to represent the ground truth, assuming perfect sensors and perfect data fusion, by applying concepts similar to CNNs. We then add heterogeneous sensors and decompose the map optimally (i.e., find which areas are best searched by which agent), and then gradually add uncertainty and reward-based trajectory optimization (i.e. reinforcement learning) while balancing exploration and exploitation. The applications of heterogeneous multi-agent searching range from agriculture to search & rescue. Future work in the field includes trying to use distributions other than the Gaussian distribution to represent more complicated sensors, and optimizing paths with fewer iterations. Keywords: Multi-robot searching, Heterogeneous, Map decomposition, Reinforcement learning, Exploration & exploitation