Large-scale typicality of Markov sample paths and consistency of MDL order estimators

Research output: Contribution to journalArticle

48 Citations (Scopus)


For Markov chains of arbitrary order, with finite alphabet A, almost sure sense limit theorems are proved on relative frequencies of k-blocks, and of symbols preceded by a given k-block, when k is permitted to grow as the sample size n grows. As an application, the consistency of two kinds of minimum description length (MDL) Markov order estimators is proved, with upper bound o(log n), respectively, α log n with α < 1/log |A|, on the permissible value of the estimated order. It was shown by Csiszár and Shields that in the absence of any bound, or with bound α log n with large α, consistency fails.

Original languageEnglish
Pages (from-to)1616-1628
Number of pages13
JournalIEEE Transactions on Information Theory
Issue number6
Publication statusPublished - Jun 1 2002



  • Conditional typicality
  • Empirical distribution
  • Iterated logarithm
  • Markov chain
  • Minimum description length (MDL)
  • Normalized maximum likelihood (NML)
  • Order estimation

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this