Numerical analysis of large Markov reward models

Miklós Telek, Sándor Rácz

Research output: Article

27 Citations (Scopus)


A first analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equations describing the process evolution in time. This paper introduces an effective numerical method for the analysis of MRMs based on the transform domain description of the system, which allows the evaluation of models with large state space (approx. 106 states). The proposed method provides the moments of reward measures on the same computational cost and memory requirement as the transient analysis of the underlying Continuous Time Markov Chain and benefits from the advantages of the randomization method, which avoids numerical instabilities and provides global error bound in advance of the computation. Implementation notes and numerical examples demonstrate the numerical properties of the proposed method are also provided.

Original languageEnglish
Pages (from-to)95-114
Number of pages20
JournalPerformance Evaluation
Publication statusPublished - aug. 1999

ASJC Scopus subject areas

  • Software
  • Modelling and Simulation
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Numerical analysis of large Markov reward models'. Together they form a unique fingerprint.

  • Cite this