Utilizing heterogeneous data sources in computational Grid workflows

Tamas Kiss, Alexandru Tudose, Gabor Terstyanszky, P. Kacsuk, Gergely Sipos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Besides computation intensive tasks, the Grid also facilitates sharing and processing very large databases and file systems that are distributed over multiple resources and administrative domains. Although accessing data in the Grid is supported by various lower level tools, end-users find it difficult to utilise these solutions directly. High level environments, such as Grid portal and workflow solutions provide little or no support for data access and manipulation. Workflow systems are widely utilised in Grid computing to automate computational tasks. Unfortunately, the ways of feeding data into these workflows is limited and in most cases requires additional tools and manual intervention. This paper describes how data can be fed into computational workflows from heterogeneous data sources. The P-GRADE Grid portal and workflow engine have been integrated with the SDSC Storage Resource Broker (SRB) in order to access SRB data resources as inputs and outputs of workflow components. The solution automates data interaction in computational workflows allowing users to seamlessly access and process data stored in SRB resources. The implemented solution also enables the seamless interoperation of SRB, SRM (Storage Resource Manager) and GridFTP file catalogues.

Original languageEnglish
Title of host publicationMaking Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments
Pages225-236
Number of pages12
Publication statusPublished - 2008
Event2007 Joint CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments - Heraklion, Crete, Greece
Duration: Jun 12 2007Jun 13 2007

Other

Other2007 Joint CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments
CountryGreece
CityHeraklion, Crete
Period6/12/076/13/07

Fingerprint

Grid computing
Managers
Engines
Processing

Keywords

  • Data management
  • Grid workflow
  • Interoperation
  • P-GRADE portal
  • SRB

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Kiss, T., Tudose, A., Terstyanszky, G., Kacsuk, P., & Sipos, G. (2008). Utilizing heterogeneous data sources in computational Grid workflows. In Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments (pp. 225-236)

Utilizing heterogeneous data sources in computational Grid workflows. / Kiss, Tamas; Tudose, Alexandru; Terstyanszky, Gabor; Kacsuk, P.; Sipos, Gergely.

Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments. 2008. p. 225-236.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kiss, T, Tudose, A, Terstyanszky, G, Kacsuk, P & Sipos, G 2008, Utilizing heterogeneous data sources in computational Grid workflows. in Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments. pp. 225-236, 2007 Joint CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments, Heraklion, Crete, Greece, 6/12/07.
Kiss T, Tudose A, Terstyanszky G, Kacsuk P, Sipos G. Utilizing heterogeneous data sources in computational Grid workflows. In Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments. 2008. p. 225-236
Kiss, Tamas ; Tudose, Alexandru ; Terstyanszky, Gabor ; Kacsuk, P. ; Sipos, Gergely. / Utilizing heterogeneous data sources in computational Grid workflows. Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments. 2008. pp. 225-236
@inproceedings{fa74af4cd9b34bfaa156d5aac6cbc8c4,
title = "Utilizing heterogeneous data sources in computational Grid workflows",
abstract = "Besides computation intensive tasks, the Grid also facilitates sharing and processing very large databases and file systems that are distributed over multiple resources and administrative domains. Although accessing data in the Grid is supported by various lower level tools, end-users find it difficult to utilise these solutions directly. High level environments, such as Grid portal and workflow solutions provide little or no support for data access and manipulation. Workflow systems are widely utilised in Grid computing to automate computational tasks. Unfortunately, the ways of feeding data into these workflows is limited and in most cases requires additional tools and manual intervention. This paper describes how data can be fed into computational workflows from heterogeneous data sources. The P-GRADE Grid portal and workflow engine have been integrated with the SDSC Storage Resource Broker (SRB) in order to access SRB data resources as inputs and outputs of workflow components. The solution automates data interaction in computational workflows allowing users to seamlessly access and process data stored in SRB resources. The implemented solution also enables the seamless interoperation of SRB, SRM (Storage Resource Manager) and GridFTP file catalogues.",
keywords = "Data management, Grid workflow, Interoperation, P-GRADE portal, SRB",
author = "Tamas Kiss and Alexandru Tudose and Gabor Terstyanszky and P. Kacsuk and Gergely Sipos",
year = "2008",
language = "English",
isbn = "9780387784472",
pages = "225--236",
booktitle = "Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments",

}

TY - GEN

T1 - Utilizing heterogeneous data sources in computational Grid workflows

AU - Kiss, Tamas

AU - Tudose, Alexandru

AU - Terstyanszky, Gabor

AU - Kacsuk, P.

AU - Sipos, Gergely

PY - 2008

Y1 - 2008

N2 - Besides computation intensive tasks, the Grid also facilitates sharing and processing very large databases and file systems that are distributed over multiple resources and administrative domains. Although accessing data in the Grid is supported by various lower level tools, end-users find it difficult to utilise these solutions directly. High level environments, such as Grid portal and workflow solutions provide little or no support for data access and manipulation. Workflow systems are widely utilised in Grid computing to automate computational tasks. Unfortunately, the ways of feeding data into these workflows is limited and in most cases requires additional tools and manual intervention. This paper describes how data can be fed into computational workflows from heterogeneous data sources. The P-GRADE Grid portal and workflow engine have been integrated with the SDSC Storage Resource Broker (SRB) in order to access SRB data resources as inputs and outputs of workflow components. The solution automates data interaction in computational workflows allowing users to seamlessly access and process data stored in SRB resources. The implemented solution also enables the seamless interoperation of SRB, SRM (Storage Resource Manager) and GridFTP file catalogues.

AB - Besides computation intensive tasks, the Grid also facilitates sharing and processing very large databases and file systems that are distributed over multiple resources and administrative domains. Although accessing data in the Grid is supported by various lower level tools, end-users find it difficult to utilise these solutions directly. High level environments, such as Grid portal and workflow solutions provide little or no support for data access and manipulation. Workflow systems are widely utilised in Grid computing to automate computational tasks. Unfortunately, the ways of feeding data into these workflows is limited and in most cases requires additional tools and manual intervention. This paper describes how data can be fed into computational workflows from heterogeneous data sources. The P-GRADE Grid portal and workflow engine have been integrated with the SDSC Storage Resource Broker (SRB) in order to access SRB data resources as inputs and outputs of workflow components. The solution automates data interaction in computational workflows allowing users to seamlessly access and process data stored in SRB resources. The implemented solution also enables the seamless interoperation of SRB, SRM (Storage Resource Manager) and GridFTP file catalogues.

KW - Data management

KW - Grid workflow

KW - Interoperation

KW - P-GRADE portal

KW - SRB

UR - http://www.scopus.com/inward/record.url?scp=84900153791&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84900153791&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9780387784472

SP - 225

EP - 236

BT - Making Grids Work - Proceedings of the CoreGRID Workshop on Programming Models Grid and P2P System Architecture Grid Systems, Tools and Environments

ER -