Stochastic Thermodynamics of Learning

Learn about our response to COVID-19, including freely available research and expanded remote access support.

You're Mobile Enabled
Access by Staats- und Universitaetsbibliothek Bremen

Stochastic Thermodynamics of Learning

Sebastian Goldt and Udo Seifert

Phys. Rev. Lett. 118, 010601 – Published 6 January 2017

Abstract

Virtually every organism gathers information about its noisy environment and builds models from those data, mostly using neural networks. Here, we use stochastic thermodynamics to analyze the learning of a classification rule by a neural network. We show that the information acquired by the network is bounded by the thermodynamic cost of learning and introduce a learning efficiency $η \leq 1$ . We discuss the conditions for optimal learning and analyze Hebbian learning in the thermodynamic limit.

Received 11 July 2016

DOI:https://doi.org/10.1103/PhysRevLett.118.010601

Physics Subject Headings (PhySH)

Learning Neuronal networks Nonequilibrium & irreversible thermodynamics Stochastic processes

Information theory

Statistical PhysicsBiological PhysicsNetworks

Authors & Affiliations

Sebastian Goldt^* and Udo Seifert

II. Institut für Theoretische Physik, Universität Stuttgart, 70550 Stuttgart, Germany

^*goldt@theo2.physik.uni-stuttgart.de

Article Text

Click to Expand

Supplemental Material

Click to Expand

References

Click to Expand

Issue

Vol. 118, Iss. 1 — 6 January 2017

Reuse & Permissions

APS and the Physical Review Editorial Office Continue to Support Researchers

COVID-19 has impacted many institutions and organizations around the world, disrupting the progress of research. Through this difficult time APS and the Physical Review editorial office are fully equipped and actively working to support researchers by continuing to carry out all editorial and peer-review functions and publish research in the journals as well as minimizing disruption to journal access.

We appreciate your continued effort and commitment to helping advance science, and allowing us to publish the best physics journals in the world. And we hope you, and your loved ones, are staying safe and healthy.

Ways to Access APS Journal Articles Off-Campus

Many researchers now find themselves working away from their institutions and, thus, may have trouble accessing the Physical Review journals. To address this, we have been improving access via several different mechanisms. See Off-Campus Access to Physical Review for further instructions.

Physical Review Letters