Robust Decentralized Mean Estimation with Limited Communication

Gábor Danner, Márk Jelasity

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Mean estimation, also known as average consensus, is an important computational primitive in decentralized systems. When the average of large vectors has to be computed, as in distributed data mining applications, reducing the communication cost becomes a key design goal. One way of reducing communication cost is to add dynamic stateful encoders and decoders to traditional mean estimation protocols. In this approach, each element of a vector message is encoded in a few bits (often only one bit) and decoded by the recipient node. However, due to this encoding and decoding mechanism, these protocols are much more sensitive to benign failure such as message drop and message delay. Properties such as mass conservation are harder to guarantee. Hence, known approaches are formulated under strong assumptions such as reliable communication, atomic non-overlapping transactions or even full synchrony. In this work, we propose a communication efficient algorithm that supports known codecs even if transactions overlap and the nodes are not synchronized. The algorithm is based on push-pull averaging, with novel features to support fault tolerance and compression. As an independent contribution, we also propose a novel codec, called the pivot codec. We demonstrate experimentally that our algorithm improves the performance of existing codecs and the novel pivot codec dominates the competing codecs in the scenarios we studied.

Original languageEnglish
Title of host publicationEuro-Par 2018
Subtitle of host publicationParallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings
EditorsMassimo Torquati, Marco Aldinucci, Luca Padovani
PublisherSpringer Verlag
Pages447-461
Number of pages15
ISBN (Print)9783319969824
DOIs
Publication statusPublished - Jan 1 2018
Event24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018 - Turin, Italy
Duration: Aug 27 2018Aug 31 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11014 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other24th International European Conference on Parallel and Distributed Computing, Euro-Par 2018
CountryItaly
CityTurin
Period8/27/188/31/18

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Danner, G., & Jelasity, M. (2018). Robust Decentralized Mean Estimation with Limited Communication. In M. Torquati, M. Aldinucci, & L. Padovani (Eds.), Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Proceedings (pp. 447-461). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11014 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-96983-1_32