Graywulf: A platform for federated scientific databases and services

László Dobos, I. Csabai, Alexander S. Szalay, Tamás Budavári, Nolan Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Many fields of science rely on relational database management systems to analyze, publish and share data. Since RDBMS are originally designed for, and their development directions are primarily driven by, business use cases they often lack features very important for scientific applications. Horizontal scalability is probably the most important miss- ing feature which makes it challenging to adapt traditional relational database systems to the ever growing data sizes. Due to the limited support of array data types and meta- data management, successful application of RDBMS in science usually requires the development of custom extensions. While some of these extensions are specific to the field of science, the majority of them could easily be generalized and reused in other disciplines. With the Graywulf project we intend to target several goals. We are building a generic platform that offers reusable components for efficient storage, transformation, statistical analysis and presentation of scientific data stored in Microsoft SQL Server. Graywulf also addresses the distributed computational issues arising from current RDBMS technologies. The current version sup- ports load balancing of simple queries and parallel execution of partitioned queries over a set of mirrored databases. Uniform user access to the data is provided through a web based query interface and a data surface for software clients. Queries are formulated in a slightly modified syntax of SQL that offers a transparent view of the distributed data. The software library consists of several components that can be reused to develop complex scientific data warehouses: a sys- tem registry, administration tools to manage entire database server clusters, a sophisticated workflow execution frame- work, and a SQL parser library.

Original languageEnglish
Title of host publicationACM International Conference Proceeding Series
DOIs
Publication statusPublished - 2013
Event25th International Conference on Scientific and Statistical Database Management, SSDBM 2013 - Baltimore, MD, United States
Duration: Jul 29 2013Jul 31 2013

Other

Other25th International Conference on Scientific and Statistical Database Management, SSDBM 2013
CountryUnited States
CityBaltimore, MD
Period7/29/137/31/13

Fingerprint

Servers
Relational database systems
Data warehouses
Metadata
Information management
Resource allocation
Scalability
Statistical methods
Industry

Keywords

  • Management

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Dobos, L., Csabai, I., Szalay, A. S., Budavári, T., & Li, N. (2013). Graywulf: A platform for federated scientific databases and services. In ACM International Conference Proceeding Series [30] https://doi.org/10.1145/2484838.2484863

Graywulf : A platform for federated scientific databases and services. / Dobos, László; Csabai, I.; Szalay, Alexander S.; Budavári, Tamás; Li, Nolan.

ACM International Conference Proceeding Series. 2013. 30.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dobos, L, Csabai, I, Szalay, AS, Budavári, T & Li, N 2013, Graywulf: A platform for federated scientific databases and services. in ACM International Conference Proceeding Series., 30, 25th International Conference on Scientific and Statistical Database Management, SSDBM 2013, Baltimore, MD, United States, 7/29/13. https://doi.org/10.1145/2484838.2484863
Dobos L, Csabai I, Szalay AS, Budavári T, Li N. Graywulf: A platform for federated scientific databases and services. In ACM International Conference Proceeding Series. 2013. 30 https://doi.org/10.1145/2484838.2484863
Dobos, László ; Csabai, I. ; Szalay, Alexander S. ; Budavári, Tamás ; Li, Nolan. / Graywulf : A platform for federated scientific databases and services. ACM International Conference Proceeding Series. 2013.
@inproceedings{9e43d39075fc4806b22b587ef5bf56c6,
title = "Graywulf: A platform for federated scientific databases and services",
abstract = "Many fields of science rely on relational database management systems to analyze, publish and share data. Since RDBMS are originally designed for, and their development directions are primarily driven by, business use cases they often lack features very important for scientific applications. Horizontal scalability is probably the most important miss- ing feature which makes it challenging to adapt traditional relational database systems to the ever growing data sizes. Due to the limited support of array data types and meta- data management, successful application of RDBMS in science usually requires the development of custom extensions. While some of these extensions are specific to the field of science, the majority of them could easily be generalized and reused in other disciplines. With the Graywulf project we intend to target several goals. We are building a generic platform that offers reusable components for efficient storage, transformation, statistical analysis and presentation of scientific data stored in Microsoft SQL Server. Graywulf also addresses the distributed computational issues arising from current RDBMS technologies. The current version sup- ports load balancing of simple queries and parallel execution of partitioned queries over a set of mirrored databases. Uniform user access to the data is provided through a web based query interface and a data surface for software clients. Queries are formulated in a slightly modified syntax of SQL that offers a transparent view of the distributed data. The software library consists of several components that can be reused to develop complex scientific data warehouses: a sys- tem registry, administration tools to manage entire database server clusters, a sophisticated workflow execution frame- work, and a SQL parser library.",
keywords = "Management",
author = "L{\'a}szl{\'o} Dobos and I. Csabai and Szalay, {Alexander S.} and Tam{\'a}s Budav{\'a}ri and Nolan Li",
year = "2013",
doi = "10.1145/2484838.2484863",
language = "English",
isbn = "9781450319218",
booktitle = "ACM International Conference Proceeding Series",

}

TY - GEN

T1 - Graywulf

T2 - A platform for federated scientific databases and services

AU - Dobos, László

AU - Csabai, I.

AU - Szalay, Alexander S.

AU - Budavári, Tamás

AU - Li, Nolan

PY - 2013

Y1 - 2013

N2 - Many fields of science rely on relational database management systems to analyze, publish and share data. Since RDBMS are originally designed for, and their development directions are primarily driven by, business use cases they often lack features very important for scientific applications. Horizontal scalability is probably the most important miss- ing feature which makes it challenging to adapt traditional relational database systems to the ever growing data sizes. Due to the limited support of array data types and meta- data management, successful application of RDBMS in science usually requires the development of custom extensions. While some of these extensions are specific to the field of science, the majority of them could easily be generalized and reused in other disciplines. With the Graywulf project we intend to target several goals. We are building a generic platform that offers reusable components for efficient storage, transformation, statistical analysis and presentation of scientific data stored in Microsoft SQL Server. Graywulf also addresses the distributed computational issues arising from current RDBMS technologies. The current version sup- ports load balancing of simple queries and parallel execution of partitioned queries over a set of mirrored databases. Uniform user access to the data is provided through a web based query interface and a data surface for software clients. Queries are formulated in a slightly modified syntax of SQL that offers a transparent view of the distributed data. The software library consists of several components that can be reused to develop complex scientific data warehouses: a sys- tem registry, administration tools to manage entire database server clusters, a sophisticated workflow execution frame- work, and a SQL parser library.

AB - Many fields of science rely on relational database management systems to analyze, publish and share data. Since RDBMS are originally designed for, and their development directions are primarily driven by, business use cases they often lack features very important for scientific applications. Horizontal scalability is probably the most important miss- ing feature which makes it challenging to adapt traditional relational database systems to the ever growing data sizes. Due to the limited support of array data types and meta- data management, successful application of RDBMS in science usually requires the development of custom extensions. While some of these extensions are specific to the field of science, the majority of them could easily be generalized and reused in other disciplines. With the Graywulf project we intend to target several goals. We are building a generic platform that offers reusable components for efficient storage, transformation, statistical analysis and presentation of scientific data stored in Microsoft SQL Server. Graywulf also addresses the distributed computational issues arising from current RDBMS technologies. The current version sup- ports load balancing of simple queries and parallel execution of partitioned queries over a set of mirrored databases. Uniform user access to the data is provided through a web based query interface and a data surface for software clients. Queries are formulated in a slightly modified syntax of SQL that offers a transparent view of the distributed data. The software library consists of several components that can be reused to develop complex scientific data warehouses: a sys- tem registry, administration tools to manage entire database server clusters, a sophisticated workflow execution frame- work, and a SQL parser library.

KW - Management

UR - http://www.scopus.com/inward/record.url?scp=84883015014&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84883015014&partnerID=8YFLogxK

U2 - 10.1145/2484838.2484863

DO - 10.1145/2484838.2484863

M3 - Conference contribution

AN - SCOPUS:84883015014

SN - 9781450319218

BT - ACM International Conference Proceeding Series

ER -