Gossip-based distributed stochastic bandit algorithms

Balázs Szörenyi, Róbert Busa-Fekete, István Hegedus, Róbert Ormándi, Márk Jelasity, Balázs Kégl

Research output: Paper

7 Citations (Scopus)

Abstract

The multi-armed bandit problem has attracted remarkable attention in the machine learning community and many efficient algorithms have been proposed to handle the so-called exploitation-exploration dilemma in various bandit setups. At the same time, significantly less effort has been devoted to adapting bandit algorithms to particular architectures, such as sensor networks, multi-core machines, or peer-to-peer (P2P) environments, which could potentially speed up their convergence. Our goal is to adapt stochastic bandit algorithms to P2P networks. In our setup, the same set of arms is available in each peer. In every iteration each peer can pull one arm independently of the other peers, and then some limited communication is possible with a few random other peers. As our main result, we show that our adaptation achieves a linear speedup in terms of the number of peers participating in the network. More precisely, we show that the probability of playing a suboptimal arm at a peer in iteration t =Ω(log N) is proportional to 1/(Nt) where N denotes the number of peers. The theoretical results are supported by simulation experiments showing that our algorithm scales gracefully with the size of network.

Original languageEnglish
Pages1056-1064
Number of pages9
Publication statusPublished - jan. 1 2013
Event30th International Conference on Machine Learning, ICML 2013 - Atlanta, GA, United States
Duration: jún. 16 2013jún. 21 2013

Other

Other30th International Conference on Machine Learning, ICML 2013
CountryUnited States
CityAtlanta, GA
Period6/16/136/21/13

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Sociology and Political Science

Fingerprint Dive into the research topics of 'Gossip-based distributed stochastic bandit algorithms'. Together they form a unique fingerprint.

  • Cite this

    Szörenyi, B., Busa-Fekete, R., Hegedus, I., Ormándi, R., Jelasity, M., & Kégl, B. (2013). Gossip-based distributed stochastic bandit algorithms. 1056-1064. Paper presented at 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, United States.