Coding for Distributed Multi-Agent Reinforcement Learning
Baoqian Wang
Junfei Xie
Nikolay Atanasov
[GitHub (To be released)]


This paper aims to mitigate straggler effects in synchronous distributed learning for multi-agent reinforcement learning (MARL) problems. Stragglers arise frequently in a distributed learning system, due to the existence of various system disturbances such as slow-downs or failures of compute nodes and communication bottlenecks. To resolve this issue, we propose a coded distributed learning framework, which speeds up the training of MARL algorithms in the presence of stragglers, while maintaining the same accuracy as the centralized approach. As an illustration, a coded distributed version of the multi-agent deep deterministic policy gradient (MADDPG) algorithm is developed and evaluated. Different coding schemes, including maximum distance separable (MDS) code, random sparse code, replication-based code, and regular low density parity check (LDPC) code are also investigated. Simulations in several multi-robot problems demonstrate the promising performance of the proposed framework.


Paper and Supplementary Material

Baoqian Wang, Junfei Xie, Nikolay Atanasov.
Coding for Distributed Multi-Agent Reinforcement Learning. (hosted on ArXiv)



We gratefully acknowledge support from National Science Foundation (NSF) under grants 1953048 and 1953049, the San Diego State University under the University Grants Program, and ARL DCIST CRA W911NF-17-2-0181. This webpage template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.