Communication-Efficient Distributed Learning Over Networks-Part I: Sufficient Conditions for Accuracy

Distributed learning is an important task in emerging applications such as localization and navigation, Internet-of-Things, and autonomous vehicles. This paper establishes a theoretical framework for learning states that evolve in real time over networks. Specifically, each agent node in the network aims to infer a time-varying state in a decentralized manner by using the node's local observations and the messages received from other nodes within its communication range. As a result, the inference accuracy of a node is significantly affected by the quality of its received messages. This calls for carefully designed strategies for generating messages that are able to provide sufficient information for the receiver and are robust to channel impairments. This paper presents communication-efficient encoding strategies for generating transmitted messages and derives a sufficient condition for the boundedness of the distributed inference error of all the agent nodes over time. The findings of this paper provide guidelines for the design of communication-efficient distributed learning in complex networked systems.