Large pseudocounts and L2-norm penalties are necessary for the mean-field inference of Ising and Potts models

    Phys. Rev. E 90, 012132 – Published 28 July 2014
    J. P. Barton, S. Cocco, E. De Leonardis, and R. Monasson

    Abstract

    The mean-field (MF) approximation offers a simple, fast way to infer direct interactions between elements in a network of correlated variables, a common, computationally challenging problem with practical applications in fields ranging from physics and biology to the social sciences. However, MF methods achieve their best performance with strong regularization, well beyond Bayesian expectations, an empirical fact that is poorly understood. In this work, we study the influence of pseudocount and L2-norm regularization schemes on the quality of inferred Ising or Potts interaction networks from correlation data within the MF approximation. We argue, based on the analysis of small systems, that the optimal value of the regularization strength remains finite even if the sampling noise tends to zero, in order to correct for systematic biases introduced by the MF approximation. Our claim is corroborated by extensive numerical studies of diverse model systems and by the analytical study of the m-component spin model for large but finite m. Additionally, we find that pseudocount regularization is robust against sampling noise and often outperforms L2-norm regularization, particularly when the underlying network of interactions is strongly heterogeneous. Much better performances are generally obtained for the Ising model than for the Potts model, for which only couplings incoming onto medium-frequency symbols are reliably inferred.

    DOI: http://dx.doi.org/10.1103/PhysRevE.90.012132

    • Figure
    • Figure
    • Figure
    • Figure
    • Figure
    • Figure
    • Figure
    11 More
    • Published 28 July 2014
    • Received 1 April 2014
    • Revised 17 June 2014

    ©2014 American Physical Society

    Authors & Affiliations

    J. P. Barton1,2, S. Cocco3, E. De Leonardis3,4, and R. Monasson5

    • 1Department of Chemical Engineering, MIT, Cambridge, Massachusetts 02139, USA
    • 2Ragon Institute of MGH, MIT and Harvard, Boston, Massachusetts 02129, USA
    • 3Laboratory of Statistical Physics of the Ecole Normale Supérieure, associated to CNRS and University P&M Curie, 24 rue Lhomond, 75005 Paris, France
    • 4UMR 7238, Computational and Quantitative Biology, UPMC Univ Paris 06, France Sorbonne Universités, 75005 Paris, France
    • 5Laboratory of Theoretical Physics of the Ecole Normale Supérieure, associated to CNRS and University P&M Curie, 24 rue Lhomond, 75005 Paris, France

    Article Text

    Click to Expand

    References

    Click to Expand

    Authorization Required


    ×

    Download & Share


    PDFExport
    ×

    Images

    2 of 18
    ×

    Log In

    Cancel
    ×

    Search


    Article Lookup
    Paste a citation or DOI

    Enter a citation
    ×