Data Bridge: Solving diverse data access in scientific Applications

Zoltán Farkas, P. Kacsuk, Ákos Balasko, Krisztián Karóczkai, Marc Santcroos, Silvia Olabarriaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The nature of data for scientific computation is very diverse in the age of big data. First, it may be available at a number of locations, e.g. the scientist's machine, some institutional filesystem, a remote service, or some sort of database. Second, the size of the data may vary from a few kilobytes to many terabytes. In order to be available for computation, data has to be transferred to the location where the computation takes place. This requires a diverse set of middleware tools that are compatible both with the data and the compute resources. However, using this tools requires additional knowledge and makes running the experiments an inconvenient task. In this paper we present the Data Bridge, a high-level service that can be used easily in scientific computations to perform data transfer to and from a diverse set of storage services. The Data Bridge not only unifies access to different types of storage services, but it can also be used at different levels (e.g., single jobs, parameter sweeps, scientific workflows) in scientific computations.

Original languageEnglish
Title of host publicationIWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways
PublisherCEUR-WS
Volume993
Publication statusPublished - 2013
Event5th International Workshop on Science Gateways, IWSG 2013 - Zurich, Switzerland
Duration: Jun 3 2013Jun 5 2013

Other

Other5th International Workshop on Science Gateways, IWSG 2013
CountrySwitzerland
CityZurich
Period6/3/136/5/13

Fingerprint

Data transfer
Middleware
Experiments
Big data

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Farkas, Z., Kacsuk, P., Balasko, Á., Karóczkai, K., Santcroos, M., & Olabarriaga, S. (2013). Data Bridge: Solving diverse data access in scientific Applications. In IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways (Vol. 993). CEUR-WS.

Data Bridge : Solving diverse data access in scientific Applications. / Farkas, Zoltán; Kacsuk, P.; Balasko, Ákos; Karóczkai, Krisztián; Santcroos, Marc; Olabarriaga, Silvia.

IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways. Vol. 993 CEUR-WS, 2013.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Farkas, Z, Kacsuk, P, Balasko, Á, Karóczkai, K, Santcroos, M & Olabarriaga, S 2013, Data Bridge: Solving diverse data access in scientific Applications. in IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways. vol. 993, CEUR-WS, 5th International Workshop on Science Gateways, IWSG 2013, Zurich, Switzerland, 6/3/13.
Farkas Z, Kacsuk P, Balasko Á, Karóczkai K, Santcroos M, Olabarriaga S. Data Bridge: Solving diverse data access in scientific Applications. In IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways. Vol. 993. CEUR-WS. 2013
Farkas, Zoltán ; Kacsuk, P. ; Balasko, Ákos ; Karóczkai, Krisztián ; Santcroos, Marc ; Olabarriaga, Silvia. / Data Bridge : Solving diverse data access in scientific Applications. IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways. Vol. 993 CEUR-WS, 2013.
@inproceedings{9c4991708a5c4c5a9c99a290b1a4bdfe,
title = "Data Bridge: Solving diverse data access in scientific Applications",
abstract = "The nature of data for scientific computation is very diverse in the age of big data. First, it may be available at a number of locations, e.g. the scientist's machine, some institutional filesystem, a remote service, or some sort of database. Second, the size of the data may vary from a few kilobytes to many terabytes. In order to be available for computation, data has to be transferred to the location where the computation takes place. This requires a diverse set of middleware tools that are compatible both with the data and the compute resources. However, using this tools requires additional knowledge and makes running the experiments an inconvenient task. In this paper we present the Data Bridge, a high-level service that can be used easily in scientific computations to perform data transfer to and from a diverse set of storage services. The Data Bridge not only unifies access to different types of storage services, but it can also be used at different levels (e.g., single jobs, parameter sweeps, scientific workflows) in scientific computations.",
author = "Zolt{\'a}n Farkas and P. Kacsuk and {\'A}kos Balasko and Kriszti{\'a}n Kar{\'o}czkai and Marc Santcroos and Silvia Olabarriaga",
year = "2013",
language = "English",
volume = "993",
booktitle = "IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways",
publisher = "CEUR-WS",

}

TY - GEN

T1 - Data Bridge

T2 - Solving diverse data access in scientific Applications

AU - Farkas, Zoltán

AU - Kacsuk, P.

AU - Balasko, Ákos

AU - Karóczkai, Krisztián

AU - Santcroos, Marc

AU - Olabarriaga, Silvia

PY - 2013

Y1 - 2013

N2 - The nature of data for scientific computation is very diverse in the age of big data. First, it may be available at a number of locations, e.g. the scientist's machine, some institutional filesystem, a remote service, or some sort of database. Second, the size of the data may vary from a few kilobytes to many terabytes. In order to be available for computation, data has to be transferred to the location where the computation takes place. This requires a diverse set of middleware tools that are compatible both with the data and the compute resources. However, using this tools requires additional knowledge and makes running the experiments an inconvenient task. In this paper we present the Data Bridge, a high-level service that can be used easily in scientific computations to perform data transfer to and from a diverse set of storage services. The Data Bridge not only unifies access to different types of storage services, but it can also be used at different levels (e.g., single jobs, parameter sweeps, scientific workflows) in scientific computations.

AB - The nature of data for scientific computation is very diverse in the age of big data. First, it may be available at a number of locations, e.g. the scientist's machine, some institutional filesystem, a remote service, or some sort of database. Second, the size of the data may vary from a few kilobytes to many terabytes. In order to be available for computation, data has to be transferred to the location where the computation takes place. This requires a diverse set of middleware tools that are compatible both with the data and the compute resources. However, using this tools requires additional knowledge and makes running the experiments an inconvenient task. In this paper we present the Data Bridge, a high-level service that can be used easily in scientific computations to perform data transfer to and from a diverse set of storage services. The Data Bridge not only unifies access to different types of storage services, but it can also be used at different levels (e.g., single jobs, parameter sweeps, scientific workflows) in scientific computations.

UR - http://www.scopus.com/inward/record.url?scp=84922572721&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84922572721&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84922572721

VL - 993

BT - IWSG 2013 - Proceedings of the 5th International Workshop on Science Gateways

PB - CEUR-WS

ER -