Coding for Distributed Multi-Agent Reinforcement Learning
Baoqian Wang
Junfei Xie
Nikolay Atanasov
[Paper]
[GitHub (To be released)]

Abstract

This paper aims to mitigate straggler effects in synchronous distributed learning for multi-agent reinforcement learning (MARL) problems. Stragglers arise frequently in a distributed learning system, due to the existence of various system disturbances such as slow-downs or failures of compute nodes and communication bottlenecks. To resolve this issue, we propose a coded distributed learning framework, which speeds up the training of MARL algorithms in the presence of stragglers, while maintaining the same accuracy as the centralized approach. As an illustration, a coded distributed version of the multi-agent deep deterministic policy gradient (MADDPG) algorithm is developed and evaluated. Different coding schemes, including maximum distance separable (MDS) code, random sparse code, replication-based code, and regular low density parity check (LDPC) code are also investigated. Simulations in several multi-robot problems demonstrate the promising performance of the proposed framework.


Presentation




Paper and Supplementary Material

Baoqian Wang, Junfei Xie, Nikolay Atanasov.
Coding for Distributed Multi-Agent Reinforcement Learning. (hosted on ArXiv)


[Bibtex]


Acknowledgements

We gratefully acknowledge support from National Science Foundation (NSF) under grants 1953048 and 1953049, the San Diego State University under the University Grants Program, and ARL DCIST CRA W911NF-17-2-0181. This webpage template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.