Fully distributed robust singular value decomposition

Istvan Hegedus, Mark Jelasity, Levente Kocsis, Andras A. Benczur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Low-rank matrix approximation is an important tool in data mining with a wide range of applications including recommender systems, clustering, and identifying topics in documents. The problem we tackle is implementing singular value decomposition (SVD)-a popular method for low rank approximationin large fully distributed P2P systems in a robust and scalable manner. We assume that the matrix to be approximated is stored in a large network where each node knows one row of the matrix (personal attributes, documents, media ratings, etc). In our P2P model, we do not allow this personal information to leave the node, yet we want the nodes to collaboratively compute the SVD. Methods applied in large scale distributed systems such as synchronized parallel gradient search or distributed iterative methods are not preferable in our system model due to their requirements of synchronized rounds or their inherent issues with load balancing. Our approach overcomes these limitations with the help of a distributed stochastic gradient search in which the personal part of the decomposition remains local, and the global part (e.g., movie features) converges at all nodes to the correct value. We present a theoretical derivation of our algorithm, as well as a thorough experimental evaluation of real and synthetic data as well. We demonstrate that the convergence speed of our method is competitive while not relying on synchronization and being robust to extreme failure scenarios.

Original languageEnglish
Title of host publication14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781479962013
DOIs
Publication statusPublished - Oct 22 2014
Event14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - London, United Kingdom
Duration: Sep 9 2014Sep 11 2014

Publication series

Name14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings

Other

Other14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014
CountryUnited Kingdom
CityLondon
Period9/9/149/11/14

    Fingerprint

Keywords

  • data mining
  • matrix factorization
  • online learning
  • privacy
  • singular value decomposition
  • stochastic gradient descent

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Hegedus, I., Jelasity, M., Kocsis, L., & Benczur, A. A. (2014). Fully distributed robust singular value decomposition. In 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings [6934299] (14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/P2P.2014.6934299