Analysis of second-order Markov reward models

G. Horváth, S. Rácz, M. Telek

Research output: Contribution to conferencePaper

4 Citations (Scopus)

Abstract

This paper considers the analysis of second-order Markov reward models. In these systems the reward accumulation during state sojourns is not deterministic, but follows a Brownian motion with a state dependent drift and variance parameter. We give the differential equations that describe the density function and the moments of the accumulated reward, and show the similarities compared to the first-order (ordinary) case. A randomization based numerical method is also presented which is numerically stable, has an error bound to control the precision, and allows the efficient analysis of large models. The computational cost of the proposed procedure is practically the same as the one of the analysis of first-order reward models, while the modeling power of second-order models is dearly larger.

Original languageEnglish
Pages845-854
Number of pages10
DOIs
Publication statusPublished - Jan 1 2004
Event2004 International Conference on Dependable Systems and Networks - Florence, Italy
Duration: Jun 28 2004Jul 1 2004

Other

Other2004 International Conference on Dependable Systems and Networks
CountryItaly
CityFlorence
Period6/28/047/1/04

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Analysis of second-order Markov reward models'. Together they form a unique fingerprint.

  • Cite this

    Horváth, G., Rácz, S., & Telek, M. (2004). Analysis of second-order Markov reward models. 845-854. Paper presented at 2004 International Conference on Dependable Systems and Networks, Florence, Italy. https://doi.org/10.1109/dsn.2004.1311955