Robust Fully Distributed Minibatch Gradient Descent with Privacy Preservation

Gábor Danner, Árpád Berta, István Hegedus, M. Jelasity

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Privacy and security are among the highest priorities in data mining approaches over data collected from mobile devices. Fully distributed machine learning is a promising direction in this context. However, it is a hard problem to design protocols that are efficient yet provide sufficient levels of privacy and security. In fully distributed environments, secure multiparty computation (MPC) is often applied to solve these problems. However, in our dynamic and unreliable application domain, known MPC algorithms are not scalable or not robust enough. We propose a light-weight protocol to quickly and securely compute the sum query over a subset of participants assuming a semihonest adversary. During the computation the participants learn no individual values. We apply this protocol to efficiently calculate the sum of gradients as part of a fully distributed minibatch stochastic gradient descent algorithm. The protocol achieves scalability and robustness by exploiting the fact that in this application domain a "quick and dirty" sum computation is acceptable. We utilize the Paillier homomorphic cryptosystem as part of our solution combined with extreme lossy gradient compression to make the cost of the cryptographic algorithms affordable. We demonstrate both theoretically and experimentally, based on churn statistics from a real smartphone trace, that the protocol is indeed practically viable.

Original languageEnglish
Article number6728020
JournalSecurity and Communication Networks
Volume2018
DOIs
Publication statusPublished - Jan 1 2018

Fingerprint

Smartphones
Mobile devices
Cryptography
Data mining
Learning systems
Scalability
Statistics
Costs

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications

Cite this

Robust Fully Distributed Minibatch Gradient Descent with Privacy Preservation. / Danner, Gábor; Berta, Árpád; Hegedus, István; Jelasity, M.

In: Security and Communication Networks, Vol. 2018, 6728020, 01.01.2018.

Research output: Contribution to journalArticle

@article{6358cf52a9b440679ace02f83cdd8153,
title = "Robust Fully Distributed Minibatch Gradient Descent with Privacy Preservation",
abstract = "Privacy and security are among the highest priorities in data mining approaches over data collected from mobile devices. Fully distributed machine learning is a promising direction in this context. However, it is a hard problem to design protocols that are efficient yet provide sufficient levels of privacy and security. In fully distributed environments, secure multiparty computation (MPC) is often applied to solve these problems. However, in our dynamic and unreliable application domain, known MPC algorithms are not scalable or not robust enough. We propose a light-weight protocol to quickly and securely compute the sum query over a subset of participants assuming a semihonest adversary. During the computation the participants learn no individual values. We apply this protocol to efficiently calculate the sum of gradients as part of a fully distributed minibatch stochastic gradient descent algorithm. The protocol achieves scalability and robustness by exploiting the fact that in this application domain a {"}quick and dirty{"} sum computation is acceptable. We utilize the Paillier homomorphic cryptosystem as part of our solution combined with extreme lossy gradient compression to make the cost of the cryptographic algorithms affordable. We demonstrate both theoretically and experimentally, based on churn statistics from a real smartphone trace, that the protocol is indeed practically viable.",
author = "G{\'a}bor Danner and {\'A}rp{\'a}d Berta and Istv{\'a}n Hegedus and M. Jelasity",
year = "2018",
month = "1",
day = "1",
doi = "10.1155/2018/6728020",
language = "English",
volume = "2018",
journal = "Security and Communication Networks",
issn = "1939-0114",
publisher = "John Wiley and Sons Inc.",

}

TY - JOUR

T1 - Robust Fully Distributed Minibatch Gradient Descent with Privacy Preservation

AU - Danner, Gábor

AU - Berta, Árpád

AU - Hegedus, István

AU - Jelasity, M.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Privacy and security are among the highest priorities in data mining approaches over data collected from mobile devices. Fully distributed machine learning is a promising direction in this context. However, it is a hard problem to design protocols that are efficient yet provide sufficient levels of privacy and security. In fully distributed environments, secure multiparty computation (MPC) is often applied to solve these problems. However, in our dynamic and unreliable application domain, known MPC algorithms are not scalable or not robust enough. We propose a light-weight protocol to quickly and securely compute the sum query over a subset of participants assuming a semihonest adversary. During the computation the participants learn no individual values. We apply this protocol to efficiently calculate the sum of gradients as part of a fully distributed minibatch stochastic gradient descent algorithm. The protocol achieves scalability and robustness by exploiting the fact that in this application domain a "quick and dirty" sum computation is acceptable. We utilize the Paillier homomorphic cryptosystem as part of our solution combined with extreme lossy gradient compression to make the cost of the cryptographic algorithms affordable. We demonstrate both theoretically and experimentally, based on churn statistics from a real smartphone trace, that the protocol is indeed practically viable.

AB - Privacy and security are among the highest priorities in data mining approaches over data collected from mobile devices. Fully distributed machine learning is a promising direction in this context. However, it is a hard problem to design protocols that are efficient yet provide sufficient levels of privacy and security. In fully distributed environments, secure multiparty computation (MPC) is often applied to solve these problems. However, in our dynamic and unreliable application domain, known MPC algorithms are not scalable or not robust enough. We propose a light-weight protocol to quickly and securely compute the sum query over a subset of participants assuming a semihonest adversary. During the computation the participants learn no individual values. We apply this protocol to efficiently calculate the sum of gradients as part of a fully distributed minibatch stochastic gradient descent algorithm. The protocol achieves scalability and robustness by exploiting the fact that in this application domain a "quick and dirty" sum computation is acceptable. We utilize the Paillier homomorphic cryptosystem as part of our solution combined with extreme lossy gradient compression to make the cost of the cryptographic algorithms affordable. We demonstrate both theoretically and experimentally, based on churn statistics from a real smartphone trace, that the protocol is indeed practically viable.

UR - http://www.scopus.com/inward/record.url?scp=85047789028&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047789028&partnerID=8YFLogxK

U2 - 10.1155/2018/6728020

DO - 10.1155/2018/6728020

M3 - Article

AN - SCOPUS:85047789028

VL - 2018

JO - Security and Communication Networks

JF - Security and Communication Networks

SN - 1939-0114

M1 - 6728020

ER -