Fully distributed robust singular value decomposition

Istvan Hegedus, M. Jelasity, Levente Kocsis, Andras A. Benczur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Low-rank matrix approximation is an important tool in data mining with a wide range of applications including recommender systems, clustering, and identifying topics in documents. The problem we tackle is implementing singular value decomposition (SVD)-a popular method for low rank approximationin large fully distributed P2P systems in a robust and scalable manner. We assume that the matrix to be approximated is stored in a large network where each node knows one row of the matrix (personal attributes, documents, media ratings, etc). In our P2P model, we do not allow this personal information to leave the node, yet we want the nodes to collaboratively compute the SVD. Methods applied in large scale distributed systems such as synchronized parallel gradient search or distributed iterative methods are not preferable in our system model due to their requirements of synchronized rounds or their inherent issues with load balancing. Our approach overcomes these limitations with the help of a distributed stochastic gradient search in which the personal part of the decomposition remains local, and the global part (e.g., movie features) converges at all nodes to the correct value. We present a theoretical derivation of our algorithm, as well as a thorough experimental evaluation of real and synthetic data as well. We demonstrate that the convergence speed of our method is competitive while not relying on synchronization and being robust to extreme failure scenarios.

Original languageEnglish
Title of host publication14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)9781479962013
DOIs
Publication statusPublished - Oct 22 2014
Event14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - London, United Kingdom
Duration: Sep 9 2014Sep 11 2014

Other

Other14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014
CountryUnited Kingdom
CityLondon
Period9/9/149/11/14

Fingerprint

Singular value decomposition
Recommender systems
Iterative methods
Resource allocation
Data mining
Synchronization

Keywords

  • data mining
  • matrix factorization
  • online learning
  • privacy
  • singular value decomposition
  • stochastic gradient descent

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Hegedus, I., Jelasity, M., Kocsis, L., & Benczur, A. A. (2014). Fully distributed robust singular value decomposition. In 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings [6934299] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/P2P.2014.6934299

Fully distributed robust singular value decomposition. / Hegedus, Istvan; Jelasity, M.; Kocsis, Levente; Benczur, Andras A.

14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2014. 6934299.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hegedus, I, Jelasity, M, Kocsis, L & Benczur, AA 2014, Fully distributed robust singular value decomposition. in 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings., 6934299, Institute of Electrical and Electronics Engineers Inc., 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014, London, United Kingdom, 9/9/14. https://doi.org/10.1109/P2P.2014.6934299
Hegedus I, Jelasity M, Kocsis L, Benczur AA. Fully distributed robust singular value decomposition. In 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2014. 6934299 https://doi.org/10.1109/P2P.2014.6934299
Hegedus, Istvan ; Jelasity, M. ; Kocsis, Levente ; Benczur, Andras A. / Fully distributed robust singular value decomposition. 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2014.
@inproceedings{728e76047f78407589a6be84683dd9e8,
title = "Fully distributed robust singular value decomposition",
abstract = "Low-rank matrix approximation is an important tool in data mining with a wide range of applications including recommender systems, clustering, and identifying topics in documents. The problem we tackle is implementing singular value decomposition (SVD)-a popular method for low rank approximationin large fully distributed P2P systems in a robust and scalable manner. We assume that the matrix to be approximated is stored in a large network where each node knows one row of the matrix (personal attributes, documents, media ratings, etc). In our P2P model, we do not allow this personal information to leave the node, yet we want the nodes to collaboratively compute the SVD. Methods applied in large scale distributed systems such as synchronized parallel gradient search or distributed iterative methods are not preferable in our system model due to their requirements of synchronized rounds or their inherent issues with load balancing. Our approach overcomes these limitations with the help of a distributed stochastic gradient search in which the personal part of the decomposition remains local, and the global part (e.g., movie features) converges at all nodes to the correct value. We present a theoretical derivation of our algorithm, as well as a thorough experimental evaluation of real and synthetic data as well. We demonstrate that the convergence speed of our method is competitive while not relying on synchronization and being robust to extreme failure scenarios.",
keywords = "data mining, matrix factorization, online learning, privacy, singular value decomposition, stochastic gradient descent",
author = "Istvan Hegedus and M. Jelasity and Levente Kocsis and Benczur, {Andras A.}",
year = "2014",
month = "10",
day = "22",
doi = "10.1109/P2P.2014.6934299",
language = "English",
isbn = "9781479962013",
booktitle = "14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Fully distributed robust singular value decomposition

AU - Hegedus, Istvan

AU - Jelasity, M.

AU - Kocsis, Levente

AU - Benczur, Andras A.

PY - 2014/10/22

Y1 - 2014/10/22

N2 - Low-rank matrix approximation is an important tool in data mining with a wide range of applications including recommender systems, clustering, and identifying topics in documents. The problem we tackle is implementing singular value decomposition (SVD)-a popular method for low rank approximationin large fully distributed P2P systems in a robust and scalable manner. We assume that the matrix to be approximated is stored in a large network where each node knows one row of the matrix (personal attributes, documents, media ratings, etc). In our P2P model, we do not allow this personal information to leave the node, yet we want the nodes to collaboratively compute the SVD. Methods applied in large scale distributed systems such as synchronized parallel gradient search or distributed iterative methods are not preferable in our system model due to their requirements of synchronized rounds or their inherent issues with load balancing. Our approach overcomes these limitations with the help of a distributed stochastic gradient search in which the personal part of the decomposition remains local, and the global part (e.g., movie features) converges at all nodes to the correct value. We present a theoretical derivation of our algorithm, as well as a thorough experimental evaluation of real and synthetic data as well. We demonstrate that the convergence speed of our method is competitive while not relying on synchronization and being robust to extreme failure scenarios.

AB - Low-rank matrix approximation is an important tool in data mining with a wide range of applications including recommender systems, clustering, and identifying topics in documents. The problem we tackle is implementing singular value decomposition (SVD)-a popular method for low rank approximationin large fully distributed P2P systems in a robust and scalable manner. We assume that the matrix to be approximated is stored in a large network where each node knows one row of the matrix (personal attributes, documents, media ratings, etc). In our P2P model, we do not allow this personal information to leave the node, yet we want the nodes to collaboratively compute the SVD. Methods applied in large scale distributed systems such as synchronized parallel gradient search or distributed iterative methods are not preferable in our system model due to their requirements of synchronized rounds or their inherent issues with load balancing. Our approach overcomes these limitations with the help of a distributed stochastic gradient search in which the personal part of the decomposition remains local, and the global part (e.g., movie features) converges at all nodes to the correct value. We present a theoretical derivation of our algorithm, as well as a thorough experimental evaluation of real and synthetic data as well. We demonstrate that the convergence speed of our method is competitive while not relying on synchronization and being robust to extreme failure scenarios.

KW - data mining

KW - matrix factorization

KW - online learning

KW - privacy

KW - singular value decomposition

KW - stochastic gradient descent

UR - http://www.scopus.com/inward/record.url?scp=84949154531&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84949154531&partnerID=8YFLogxK

U2 - 10.1109/P2P.2014.6934299

DO - 10.1109/P2P.2014.6934299

M3 - Conference contribution

SN - 9781479962013

BT - 14th IEEE International Conference on Peer-to-Peer Computing, IEEE P2P 2014 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -