Large-scale typicality of Markov sample paths and consistency of MDL order estimators

Research output: Contribution to journalArticle

48 Citations (Scopus)

Abstract

For Markov chains of arbitrary order, with finite alphabet A, almost sure sense limit theorems are proved on relative frequencies of k-blocks, and of symbols preceded by a given k-block, when k is permitted to grow as the sample size n grows. As an application, the consistency of two kinds of minimum description length (MDL) Markov order estimators is proved, with upper bound o(log n), respectively, α log n with α <1/log |A|, on the permissible value of the estimated order. It was shown by Csiszár and Shields that in the absence of any bound, or with bound α log n with large α, consistency fails.

Original languageEnglish
Pages (from-to)1616-1628
Number of pages13
JournalIEEE Transactions on Information Theory
Volume48
Issue number6
DOIs
Publication statusPublished - Jun 2002

Fingerprint

Markov processes
symbol
Values

Keywords

  • Conditional typicality
  • Empirical distribution
  • Iterated logarithm
  • Markov chain
  • Minimum description length (MDL)
  • Normalized maximum likelihood (NML)
  • Order estimation

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Information Systems

Cite this

Large-scale typicality of Markov sample paths and consistency of MDL order estimators. / Csiszár, I.

In: IEEE Transactions on Information Theory, Vol. 48, No. 6, 06.2002, p. 1616-1628.

Research output: Contribution to journalArticle

@article{d6c03b5f836c494e9bcd70f4b6f7fe25,
title = "Large-scale typicality of Markov sample paths and consistency of MDL order estimators",
abstract = "For Markov chains of arbitrary order, with finite alphabet A, almost sure sense limit theorems are proved on relative frequencies of k-blocks, and of symbols preceded by a given k-block, when k is permitted to grow as the sample size n grows. As an application, the consistency of two kinds of minimum description length (MDL) Markov order estimators is proved, with upper bound o(log n), respectively, α log n with α <1/log |A|, on the permissible value of the estimated order. It was shown by Csisz{\'a}r and Shields that in the absence of any bound, or with bound α log n with large α, consistency fails.",
keywords = "Conditional typicality, Empirical distribution, Iterated logarithm, Markov chain, Minimum description length (MDL), Normalized maximum likelihood (NML), Order estimation",
author = "I. Csisz{\'a}r",
year = "2002",
month = "6",
doi = "10.1109/TIT.2002.1003842",
language = "English",
volume = "48",
pages = "1616--1628",
journal = "IEEE Transactions on Information Theory",
issn = "0018-9448",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

TY - JOUR

T1 - Large-scale typicality of Markov sample paths and consistency of MDL order estimators

AU - Csiszár, I.

PY - 2002/6

Y1 - 2002/6

N2 - For Markov chains of arbitrary order, with finite alphabet A, almost sure sense limit theorems are proved on relative frequencies of k-blocks, and of symbols preceded by a given k-block, when k is permitted to grow as the sample size n grows. As an application, the consistency of two kinds of minimum description length (MDL) Markov order estimators is proved, with upper bound o(log n), respectively, α log n with α <1/log |A|, on the permissible value of the estimated order. It was shown by Csiszár and Shields that in the absence of any bound, or with bound α log n with large α, consistency fails.

AB - For Markov chains of arbitrary order, with finite alphabet A, almost sure sense limit theorems are proved on relative frequencies of k-blocks, and of symbols preceded by a given k-block, when k is permitted to grow as the sample size n grows. As an application, the consistency of two kinds of minimum description length (MDL) Markov order estimators is proved, with upper bound o(log n), respectively, α log n with α <1/log |A|, on the permissible value of the estimated order. It was shown by Csiszár and Shields that in the absence of any bound, or with bound α log n with large α, consistency fails.

KW - Conditional typicality

KW - Empirical distribution

KW - Iterated logarithm

KW - Markov chain

KW - Minimum description length (MDL)

KW - Normalized maximum likelihood (NML)

KW - Order estimation

UR - http://www.scopus.com/inward/record.url?scp=0036611605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036611605&partnerID=8YFLogxK

U2 - 10.1109/TIT.2002.1003842

DO - 10.1109/TIT.2002.1003842

M3 - Article

AN - SCOPUS:0036611605

VL - 48

SP - 1616

EP - 1628

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 6

ER -