Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling

Abstract

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k th$ order Markov chains, for arbitrary $k$ , from finite data by applying Bayesian methods to both parameter estimation and model-order selection. Extending existing results for multinomial models of discrete data, we connect inference to statistical mechanics through information-theoretic (type theory) techniques. We establish a direct relationship between Bayesian evidence and the partition function which allows for straightforward calculation of the expectation and variance of the conditional relative entropy and the source entropy rate. Finally, we introduce a method that uses finite data-size scaling with model-order comparison to infer the structure of out-of-class processes.

6 More

Received 25 March 2007
Publisher error corrected 18 July 2007

DOI:

Erratum

Publisher's Note: Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling [Phys. Rev. E 76, 011106 (2007)]

Christopher C. Strelioff, James P. Crutchfield, and Alfred W. Hübler

Phys. Rev. E 76, 019902 (2007)

Authors & Affiliations

Christopher C. Strelioff^1,2,*, James P. Crutchfield^1,†, and Alfred W. Hübler^2,‡

¹Center for Computational Science & Engineering and Physics Department, University of California at Davis, One Shields Avenue, Davis, California 95616, USA
²Center for Complex Systems Research and Physics Department, University of Illinois at Urbana-Champaign, 1110 West Green Street, Urbana, Illinois 61801, USA

^*streliof@uiuc.edu
^†chaos@cse.ucdavis.edu
^‡a-hubler@uiuc.edu

Article Text

Click to Expand

References

Click to Expand

Physical Review E

statistical, nonlinear, and soft matter physics