Towards scalable pattern-based optimization for dense linear algebra

D. Berényi, András Leitereg, Gábor Lehel

Research output: Article

Abstract

Linear algebraic expressions are the essence of many computationally intensive problems, including scientific simulations and machine learning applications. However, translating high-level formulations of these expressions to efficient machine-level representations is far from trivial: developers should be assisted by automatic optimization tools so that they can focus their attention on high-level problems, rather than low-level details. The tractability of these optimizations is highly dependent on the choice of the primitive constructs in terms of which the computations are to be expressed. In this work, we propose to describe operations on multi-dimensional arrays using a selection of higher-order functions, inspired by functional programming, and we present rewrite rules for these such that they can be automatically optimized for modern hierarchical and heterogeneous architectures. Using this formalism, we systematically construct and analyze different subdivisions and permutations of the dense matrix multiplication problem.

Original languageEnglish
JournalConcurrency Computation
DOIs
Publication statusAccepted/In press - jan. 1 2018

Fingerprint

Linear algebra
Functional programming
Optimization
Multidimensional Arrays
Learning systems
Functional Programming
Tractability
Matrix multiplication
Subdivision
Machine Learning
Permutation
Trivial
Higher Order
Formulation
Dependent
Simulation

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Computer Science Applications
  • Computer Networks and Communications
  • Computational Theory and Mathematics

Cite this

Towards scalable pattern-based optimization for dense linear algebra. / Berényi, D.; Leitereg, András; Lehel, Gábor.

In: Concurrency Computation, 01.01.2018.

Research output: Article

@article{54f739365e6144979fc81461f5f68fca,
title = "Towards scalable pattern-based optimization for dense linear algebra",
abstract = "Linear algebraic expressions are the essence of many computationally intensive problems, including scientific simulations and machine learning applications. However, translating high-level formulations of these expressions to efficient machine-level representations is far from trivial: developers should be assisted by automatic optimization tools so that they can focus their attention on high-level problems, rather than low-level details. The tractability of these optimizations is highly dependent on the choice of the primitive constructs in terms of which the computations are to be expressed. In this work, we propose to describe operations on multi-dimensional arrays using a selection of higher-order functions, inspired by functional programming, and we present rewrite rules for these such that they can be automatically optimized for modern hierarchical and heterogeneous architectures. Using this formalism, we systematically construct and analyze different subdivisions and permutations of the dense matrix multiplication problem.",
keywords = "functional programming, higher-order function fusion, optimization patterns, rewrite rules",
author = "D. Ber{\'e}nyi and Andr{\'a}s Leitereg and G{\'a}bor Lehel",
year = "2018",
month = "1",
day = "1",
doi = "10.1002/cpe.4696",
language = "English",
journal = "Concurrency Computation Practice and Experience",
issn = "1532-0626",
publisher = "John Wiley and Sons Ltd",

}

TY - JOUR

T1 - Towards scalable pattern-based optimization for dense linear algebra

AU - Berényi, D.

AU - Leitereg, András

AU - Lehel, Gábor

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Linear algebraic expressions are the essence of many computationally intensive problems, including scientific simulations and machine learning applications. However, translating high-level formulations of these expressions to efficient machine-level representations is far from trivial: developers should be assisted by automatic optimization tools so that they can focus their attention on high-level problems, rather than low-level details. The tractability of these optimizations is highly dependent on the choice of the primitive constructs in terms of which the computations are to be expressed. In this work, we propose to describe operations on multi-dimensional arrays using a selection of higher-order functions, inspired by functional programming, and we present rewrite rules for these such that they can be automatically optimized for modern hierarchical and heterogeneous architectures. Using this formalism, we systematically construct and analyze different subdivisions and permutations of the dense matrix multiplication problem.

AB - Linear algebraic expressions are the essence of many computationally intensive problems, including scientific simulations and machine learning applications. However, translating high-level formulations of these expressions to efficient machine-level representations is far from trivial: developers should be assisted by automatic optimization tools so that they can focus their attention on high-level problems, rather than low-level details. The tractability of these optimizations is highly dependent on the choice of the primitive constructs in terms of which the computations are to be expressed. In this work, we propose to describe operations on multi-dimensional arrays using a selection of higher-order functions, inspired by functional programming, and we present rewrite rules for these such that they can be automatically optimized for modern hierarchical and heterogeneous architectures. Using this formalism, we systematically construct and analyze different subdivisions and permutations of the dense matrix multiplication problem.

KW - functional programming

KW - higher-order function fusion

KW - optimization patterns

KW - rewrite rules

UR - http://www.scopus.com/inward/record.url?scp=85052968112&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052968112&partnerID=8YFLogxK

U2 - 10.1002/cpe.4696

DO - 10.1002/cpe.4696

M3 - Article

AN - SCOPUS:85052968112

JO - Concurrency Computation Practice and Experience

JF - Concurrency Computation Practice and Experience

SN - 1532-0626

ER -