Utilizing heterogeneous data sources in computational Grid workflows

Tamas Kiss, Alexandru Tudose, Gabor Terstyanszky, Peter Kacsuk, Gergely Sipos

Research output: Conference contribution

1 Citation (Scopus)

Abstract

Besides computation intensive tasks, the Grid also facilitates sharing and processing very large databases and file systems that are distributed over multiple resources and administrative domains. Although accessing data in the Grid is supported by various lower level tools, end-users find it difficult to utilise these solutions directly. High level environments, such as Grid portal and workflow solutions provide little or no support for data access and manipulation. Workflow systems are widely utilised in Grid computing to automate computational tasks. Unfortunately, the ways of feeding data into these workflows is limited and in most cases requires additional tools and manual intervention. This paper describes how data can be fed into computational workflows from heterogeneous data sources. The P-GRADE Grid portal and workflow engine have been integrated with the SDSC Storage Resource Broker (SRB) in order to access SRB data resources as inputs and outputs of workflow components. The solution automates data interaction in computational workflows allowing users to seamlessly access and process data stored in SRB resources. The implemented solution also enables the seamless interoperation of SRB, SRM (Storage Resource Manager) and GridFTP file catalogues.

Original languageEnglish
Title of host publicationMaking Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments
PublisherSpringer Science and Business Media, LLC
Pages225-236
Number of pages12
ISBN (Print)9780387784472
DOIs
Publication statusPublished - 2008
Event2007 Joint CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments - Heraklion, Crete, Greece
Duration: jún. 12 2007jún. 13 2007

Publication series

NameMaking Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments

Other

Other2007 Joint CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments
CountryGreece
CityHeraklion, Crete
Period6/12/076/13/07

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Utilizing heterogeneous data sources in computational Grid workflows'. Together they form a unique fingerprint.

  • Cite this

    Kiss, T., Tudose, A., Terstyanszky, G., Kacsuk, P., & Sipos, G. (2008). Utilizing heterogeneous data sources in computational Grid workflows. In Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments (pp. 225-236). (Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments). Springer Science and Business Media, LLC. https://doi.org/10.1007/978-0-387-78448-9_18