Context tree estimation for not necessarily finite memory processes, Via BIC and MDL

Imre Csiszár, Zsolt Talata

Research output: Contribution to journalArticle

90 Citations (Scopus)


The concept of context tree, usually defined for finite memory processes, is extended to arbitrary stationary ergodic processes (with finite alphabet). These context trees are not necessarily complete, and may be of infinite depth. The familiar Bayesian information criterion (BIC) and minimum description length (MDL) principles arc shown to provide strongly consistent estimators of the context tree, via optimization of a criterion for hypothetical context trees of finite depth, allowed to grow with the sample size n as o(log n). Algorithms are provided to compute these estimators in O(n) time, and to compute them on-line for all i ≤ n in o(n log n) time.

Original languageEnglish
Pages (from-to)1007-1016
Number of pages10
JournalIEEE Transactions on Information Theory
Issue number3
Publication statusPublished - Mar 1 2006



  • Bayesian information criterion (BIG)
  • Consistent estimation
  • Context tree
  • Context tree maximization (CTM)
  • Infinite memory
  • Minimum description length (MDL)
  • Model selection

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this