Efficient reliability in volunteer storage systems with random linear coding

Visegrádi, Péter Kacsuk

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Volunteer systems pose difficult challenges for data storage. Because of the extremely low reliability of volunteer nodes, these systems require so high redundancy that replication is infeasible. Erasure coding has been proposed to cope with this problem as it needs much less redundancy to achieve the same reliability. Its downside is that the reparation of the system creates high overhead, as fully decoding the original data is required to generate new coded data.

Random linear coding has been proposed to be used as a data storage method, as it provides a better redundancy/reliability ratio, and less control overhead. We propose that it also helps in the reparation of the system, as decoding is not required; instead, coded data can be generated from already existing coded data. However, it may be possible that this iterative reparation leads to degradation of data over time; even more so, if sparse coding is used to increase compute efficiency.

This paper examines the effects of random linear coding and the iterative reparation of the system. It shows the reliability that can be achieved with random linear coding in a highly volatile distributed system. We conclude that random linear coding can achieve high reliability even in highly volatile systems.

Original languageEnglish
Title of host publicationEuro-Par 2014
Subtitle of host publicationParallel Processing Workshops - Euro-Par 2014 International Workshops, Revised Selected Papers
EditorsLuís Lopes
PublisherSpringer Verlag
Pages559-569
Number of pages11
EditionPart 1
ISBN (Electronic)9783319143248
DOIs
Publication statusPublished - Jan 1 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPart 1
Volume8805
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Efficient reliability in volunteer storage systems with random linear coding'. Together they form a unique fingerprint.

  • Cite this

    Visegrádi, & Kacsuk, P. (2014). Efficient reliability in volunteer storage systems with random linear coding. In L. Lopes (Ed.), Euro-Par 2014: Parallel Processing Workshops - Euro-Par 2014 International Workshops, Revised Selected Papers (Part 1 ed., pp. 559-569). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8805, No. Part 1). Springer Verlag. https://doi.org/10.1007/978-3-319-14325-5_48