Numerical analysis of large Markov reward models

M. Telek, Sándor Rácz

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

A first analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equations describing the process evolution in time. This paper introduces an effective numerical method for the analysis of MRMs based on the transform domain description of the system, which allows the evaluation of models with large state space (approx. 106 states). The proposed method provides the moments of reward measures on the same computational cost and memory requirement as the transient analysis of the underlying Continuous Time Markov Chain and benefits from the advantages of the randomization method, which avoids numerical instabilities and provides global error bound in advance of the computation. Implementation notes and numerical examples demonstrate the numerical properties of the proposed method are also provided.

Original languageEnglish
Pages (from-to)95-114
Number of pages20
JournalPerformance Evaluation
Volume36-37
DOIs
Publication statusPublished - Aug 1999

Fingerprint

Reward
Numerical analysis
Numerical Analysis
Numerical methods
Numerical Methods
Global Error Bound
Transform
Numerical Instability
Transient Analysis
Continuous-time Markov Chain
Randomisation
Transient analysis
Markov processes
Partial differential equations
Computational Cost
Time Domain
State Space
Partial differential equation
Numerical Solution
Model

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software
  • Modelling and Simulation
  • Statistics and Probability

Cite this

Numerical analysis of large Markov reward models. / Telek, M.; Rácz, Sándor.

In: Performance Evaluation, Vol. 36-37, 08.1999, p. 95-114.

Research output: Contribution to journalArticle

Telek, M. ; Rácz, Sándor. / Numerical analysis of large Markov reward models. In: Performance Evaluation. 1999 ; Vol. 36-37. pp. 95-114.
@article{663df2d2383346289907afa694029b2a,
title = "Numerical analysis of large Markov reward models",
abstract = "A first analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equations describing the process evolution in time. This paper introduces an effective numerical method for the analysis of MRMs based on the transform domain description of the system, which allows the evaluation of models with large state space (approx. 106 states). The proposed method provides the moments of reward measures on the same computational cost and memory requirement as the transient analysis of the underlying Continuous Time Markov Chain and benefits from the advantages of the randomization method, which avoids numerical instabilities and provides global error bound in advance of the computation. Implementation notes and numerical examples demonstrate the numerical properties of the proposed method are also provided.",
author = "M. Telek and S{\'a}ndor R{\'a}cz",
year = "1999",
month = "8",
doi = "10.1016/S0166-5316(99)00032-2",
language = "English",
volume = "36-37",
pages = "95--114",
journal = "Performance Evaluation",
issn = "0166-5316",
publisher = "Elsevier",

}

TY - JOUR

T1 - Numerical analysis of large Markov reward models

AU - Telek, M.

AU - Rácz, Sándor

PY - 1999/8

Y1 - 1999/8

N2 - A first analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equations describing the process evolution in time. This paper introduces an effective numerical method for the analysis of MRMs based on the transform domain description of the system, which allows the evaluation of models with large state space (approx. 106 states). The proposed method provides the moments of reward measures on the same computational cost and memory requirement as the transient analysis of the underlying Continuous Time Markov Chain and benefits from the advantages of the randomization method, which avoids numerical instabilities and provides global error bound in advance of the computation. Implementation notes and numerical examples demonstrate the numerical properties of the proposed method are also provided.

AB - A first analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equations describing the process evolution in time. This paper introduces an effective numerical method for the analysis of MRMs based on the transform domain description of the system, which allows the evaluation of models with large state space (approx. 106 states). The proposed method provides the moments of reward measures on the same computational cost and memory requirement as the transient analysis of the underlying Continuous Time Markov Chain and benefits from the advantages of the randomization method, which avoids numerical instabilities and provides global error bound in advance of the computation. Implementation notes and numerical examples demonstrate the numerical properties of the proposed method are also provided.

UR - http://www.scopus.com/inward/record.url?scp=0033169497&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033169497&partnerID=8YFLogxK

U2 - 10.1016/S0166-5316(99)00032-2

DO - 10.1016/S0166-5316(99)00032-2

M3 - Article

VL - 36-37

SP - 95

EP - 114

JO - Performance Evaluation

JF - Performance Evaluation

SN - 0166-5316

ER -