JA-BE-JA: A distributed algorithm for balanced graph Partitioning

Fatemeh Rahimian, Amir H. Payberah, Sarunas Girdzijauskas, M. Jelasity, Seif Haridi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

50 Citations (Scopus)

Abstract

Balanced graph partitioning is a well known NP-complete problem with a wide range of applications. These applications include many large-scale distributed problems including the optimal storage of large sets of graph-structured data over several hosts-A key problem in today's Cloud infrastructure. However, in very large-scale distributed scenarios, state-of-the-Art algorithms are not directly applicable, because they typically involve frequent global operations over the entire graph. In this paper, we propose a fully distributed algorithm, called JA-BE-JA, that uses local search and simulated annealing techniques for graph partitioning. The algorithm is massively parallel: there is no central coordination, each node is processed independently, and only the direct neighbors of the node, and a small subset of random nodes in the graph need to be known locally. Strict synchronization is not required. These features allow JA-BE-JA to be easily adapted to any distributed graph-processing system from data centers to fully distributed networks. We perform a thorough experimental analysis, which shows that the minimal edge-cut value achieved by JA-BE-JA is comparable to state-of-the-Art centralized algorithms such as METIS. In particular, on large social networks JA-BEJA outperforms METIS, which makes JA-BE-JA-A bottom-up, self-organizing algorithm-A highly competitive practical solution for graph partitioning.

Original languageEnglish
Title of host publicationInternational Conference on Self-Adaptive and Self-Organizing Systems, SASO
Pages51-60
Number of pages10
DOIs
Publication statusPublished - 2013
Event2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2013 - Philadelphia, PA, United States
Duration: Sep 9 2013Sep 13 2013

Other

Other2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2013
CountryUnited States
CityPhiladelphia, PA
Period9/9/139/13/13

Fingerprint

Parallel algorithms
Simulated annealing
Computational complexity
Synchronization
Processing

Keywords

  • distributed algorithm
  • graph partitioning
  • load balancing
  • simulated annealing

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Control and Systems Engineering

Cite this

Rahimian, F., Payberah, A. H., Girdzijauskas, S., Jelasity, M., & Haridi, S. (2013). JA-BE-JA: A distributed algorithm for balanced graph Partitioning. In International Conference on Self-Adaptive and Self-Organizing Systems, SASO (pp. 51-60). [6676492] https://doi.org/10.1109/SASO.2013.13

JA-BE-JA : A distributed algorithm for balanced graph Partitioning. / Rahimian, Fatemeh; Payberah, Amir H.; Girdzijauskas, Sarunas; Jelasity, M.; Haridi, Seif.

International Conference on Self-Adaptive and Self-Organizing Systems, SASO. 2013. p. 51-60 6676492.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rahimian, F, Payberah, AH, Girdzijauskas, S, Jelasity, M & Haridi, S 2013, JA-BE-JA: A distributed algorithm for balanced graph Partitioning. in International Conference on Self-Adaptive and Self-Organizing Systems, SASO., 6676492, pp. 51-60, 2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2013, Philadelphia, PA, United States, 9/9/13. https://doi.org/10.1109/SASO.2013.13
Rahimian F, Payberah AH, Girdzijauskas S, Jelasity M, Haridi S. JA-BE-JA: A distributed algorithm for balanced graph Partitioning. In International Conference on Self-Adaptive and Self-Organizing Systems, SASO. 2013. p. 51-60. 6676492 https://doi.org/10.1109/SASO.2013.13
Rahimian, Fatemeh ; Payberah, Amir H. ; Girdzijauskas, Sarunas ; Jelasity, M. ; Haridi, Seif. / JA-BE-JA : A distributed algorithm for balanced graph Partitioning. International Conference on Self-Adaptive and Self-Organizing Systems, SASO. 2013. pp. 51-60
@inproceedings{4df7c5ec870642b8835296f16a39b201,
title = "JA-BE-JA: A distributed algorithm for balanced graph Partitioning",
abstract = "Balanced graph partitioning is a well known NP-complete problem with a wide range of applications. These applications include many large-scale distributed problems including the optimal storage of large sets of graph-structured data over several hosts-A key problem in today's Cloud infrastructure. However, in very large-scale distributed scenarios, state-of-the-Art algorithms are not directly applicable, because they typically involve frequent global operations over the entire graph. In this paper, we propose a fully distributed algorithm, called JA-BE-JA, that uses local search and simulated annealing techniques for graph partitioning. The algorithm is massively parallel: there is no central coordination, each node is processed independently, and only the direct neighbors of the node, and a small subset of random nodes in the graph need to be known locally. Strict synchronization is not required. These features allow JA-BE-JA to be easily adapted to any distributed graph-processing system from data centers to fully distributed networks. We perform a thorough experimental analysis, which shows that the minimal edge-cut value achieved by JA-BE-JA is comparable to state-of-the-Art centralized algorithms such as METIS. In particular, on large social networks JA-BEJA outperforms METIS, which makes JA-BE-JA-A bottom-up, self-organizing algorithm-A highly competitive practical solution for graph partitioning.",
keywords = "distributed algorithm, graph partitioning, load balancing, simulated annealing",
author = "Fatemeh Rahimian and Payberah, {Amir H.} and Sarunas Girdzijauskas and M. Jelasity and Seif Haridi",
year = "2013",
doi = "10.1109/SASO.2013.13",
language = "English",
isbn = "9780769551296",
pages = "51--60",
booktitle = "International Conference on Self-Adaptive and Self-Organizing Systems, SASO",

}

TY - GEN

T1 - JA-BE-JA

T2 - A distributed algorithm for balanced graph Partitioning

AU - Rahimian, Fatemeh

AU - Payberah, Amir H.

AU - Girdzijauskas, Sarunas

AU - Jelasity, M.

AU - Haridi, Seif

PY - 2013

Y1 - 2013

N2 - Balanced graph partitioning is a well known NP-complete problem with a wide range of applications. These applications include many large-scale distributed problems including the optimal storage of large sets of graph-structured data over several hosts-A key problem in today's Cloud infrastructure. However, in very large-scale distributed scenarios, state-of-the-Art algorithms are not directly applicable, because they typically involve frequent global operations over the entire graph. In this paper, we propose a fully distributed algorithm, called JA-BE-JA, that uses local search and simulated annealing techniques for graph partitioning. The algorithm is massively parallel: there is no central coordination, each node is processed independently, and only the direct neighbors of the node, and a small subset of random nodes in the graph need to be known locally. Strict synchronization is not required. These features allow JA-BE-JA to be easily adapted to any distributed graph-processing system from data centers to fully distributed networks. We perform a thorough experimental analysis, which shows that the minimal edge-cut value achieved by JA-BE-JA is comparable to state-of-the-Art centralized algorithms such as METIS. In particular, on large social networks JA-BEJA outperforms METIS, which makes JA-BE-JA-A bottom-up, self-organizing algorithm-A highly competitive practical solution for graph partitioning.

AB - Balanced graph partitioning is a well known NP-complete problem with a wide range of applications. These applications include many large-scale distributed problems including the optimal storage of large sets of graph-structured data over several hosts-A key problem in today's Cloud infrastructure. However, in very large-scale distributed scenarios, state-of-the-Art algorithms are not directly applicable, because they typically involve frequent global operations over the entire graph. In this paper, we propose a fully distributed algorithm, called JA-BE-JA, that uses local search and simulated annealing techniques for graph partitioning. The algorithm is massively parallel: there is no central coordination, each node is processed independently, and only the direct neighbors of the node, and a small subset of random nodes in the graph need to be known locally. Strict synchronization is not required. These features allow JA-BE-JA to be easily adapted to any distributed graph-processing system from data centers to fully distributed networks. We perform a thorough experimental analysis, which shows that the minimal edge-cut value achieved by JA-BE-JA is comparable to state-of-the-Art centralized algorithms such as METIS. In particular, on large social networks JA-BEJA outperforms METIS, which makes JA-BE-JA-A bottom-up, self-organizing algorithm-A highly competitive practical solution for graph partitioning.

KW - distributed algorithm

KW - graph partitioning

KW - load balancing

KW - simulated annealing

UR - http://www.scopus.com/inward/record.url?scp=84893207551&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893207551&partnerID=8YFLogxK

U2 - 10.1109/SASO.2013.13

DO - 10.1109/SASO.2013.13

M3 - Conference contribution

AN - SCOPUS:84893207551

SN - 9780769551296

SP - 51

EP - 60

BT - International Conference on Self-Adaptive and Self-Organizing Systems, SASO

ER -