Exit frequency matrices for finite Markov chains

Andrew Beveridge, László Lovász

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Consider a finite irreducible Markov chain on state space S with transition matrix M and stationary distribution π. Let Rii = 1/πiabe the diagonal matrix of return times, Rii = 1/i. Given distributions σ, τ and k ∈ S, the exit frequency x k(σ τ) denotes the expected number of times a random walk exits state k before an optimal stopping rule from to halts the walk. For a target distribution τ, we define Xt as the n × n matrix given by (Xτ)ij = xj(i,τ), where i also denotes the singleton distribution on state i. The dual Markov chain with transition matrix = Ř MτR-1 is called the reverse chain. We prove that Markov chain duality extends to matrices of exit frequencies. Specifically, for each target distribution τ*, we associate a unique dual distribution τ*. Let X̌τ* denote the matrix of exit frequencies from singletons to τ* on the reverse chain. We show that X̌τ* = R (Xττ-b τ1)R-1, where b is a non-negative constant vector (depending on τ ). We explore this exit frequency duality and further illuminate the relationship between stopping rules on the original chain and reverse chain.

Original languageEnglish
Pages (from-to)541-560
Number of pages20
JournalCombinatorics Probability and Computing
Volume19
Issue number4
DOIs
Publication statusPublished - Jul 1 2010

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Statistics and Probability
  • Computational Theory and Mathematics
  • Applied Mathematics

Cite this