Phys. Rev. Lett. 108, 258701 (2012) - Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy

Loading [MathJax]/jax/output/CommonHTML/fonts/TeX/AMS-Regular.js

Go Mobile »
Access by University of Bath

Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy

Jakob Runge, Jobst Heitzig, Vladimir Petoukhov, and Jürgen Kurths

Phys. Rev. Lett. 108, 258701 – Published 21 June 2012

Abstract

Multivariate transfer entropy (TE) is a model-free approach to detect causalities in multivariate time series. It is able to distinguish direct from indirect causality and common drivers without assuming any underlying model. But despite these advantages it has mostly been applied in a bivariate setting as it is hard to estimate reliably in high dimensions since its definition involves infinite vectors. To overcome this limitation, we propose to embed TE into the framework of graphical models and present a formula that decomposes TE into a sum of finite-dimensional contributions that we call decomposed transfer entropy. Graphical models further provide a richer picture because they also yield the causal coupling delays. To estimate the graphical model we suggest an iterative algorithm, a modified version of the PC-algorithm with a very low estimation dimension. We present an appropriate significance test and demonstrate the method’s performance using examples of nonlinear stochastic delay-differential equations and observational climate data (sea level pressure).

Received 18 February 2012

DOI:https://doi.org/10.1103/PhysRevLett.108.258701

© 2012 American Physical Society

Authors & Affiliations

Jakob Runge^1,2, Jobst Heitzig¹, Vladimir Petoukhov¹, and Jürgen Kurths^1,2,3

¹Potsdam Institute for Climate Impact Research (PIK), 14473 Potsdam, Germany
²Department of Physics, Humboldt University, 12489 Berlin, Germany
³Institute for Complex Systems and Mathematical Biology, University of Aberdeen, Aberdeen AB24 3UE, United Kingdom

Article Text

Click to Expand

Supplemental Material

Click to Expand

References

Click to Expand

Issue

Vol. 108, Iss. 25 — 22 June 2012

Reuse & Permissions

Announcement

Information on SCOAP³ and Physical Review journals

January 3, 2018

High Energy Physics (HEP) papers published after January 1, 2018 in Physical Review Letters, Physical Review C, and Physical Review D are published open access, paid for centrally by SCOAP³. Library subscriptions will be modified accordingly. This arrangement will initially last for two years, up to the end of 2019.

Images

Figure 1
Estimated TE between all subprocesses $X$ $X$ , $Y$ $Y$ , $Z$ $Z$ , $W$ $W$ for an example process [a discrete-time sampling of Eq. (5) that will be analyzed later] with true coupling structure given by the black arrows. If we truncate the infinite past vectors used in the common definition of TE at 15 lags, the estimation dimension is 61. The true TE of all gray and red links is zero, since they do not represent direct couplings. So of the four estimates of comparable size marked in red boxes, only the link $Y \to W$ $Y \to W$ is a correctly identified coupling.Reuse & Permissions
Figure 2
Causal interactions in a multivariate process $X$ $X$ . (a) Time series graph (see definition in text). The boxes mark different sets of conditions for the CMI between $X_{t - τ}$ $X_{t - τ}$ at $τ = 2$ $τ = 2$ and $Y_{t}$ $Y_{t}$ (marked by the black dots): the terms in Eq. (1) use the infinite set $X_{t}^{-} ∖ X_{t}^{-} \cup X_{t - τ}^{-}$ $X_{t}^{-} ∖ X_{t}^{-} \cup X_{t - τ}^{-}$ (gray dotted line). But only the finite set $S_{Y_{t}, X_{t - τ}}$ (red dashed line) is needed to satisfy Eq. (2). $S_{Y_{t}, X_{t - τ}}$ must be chosen so that it separates the skipped infinite conditions $(X_{t}^{-} ∖ X_{t}^{-} \cup X_{t - τ}^{-}) ∖ S_{Y_{t}, X_{t - τ}}$ from $Y_{t}$ in the graph (for a formal definition of paths and separation, see 13). The even smaller set of parents $P_{Y_{t}}$ (blue box, gray nodes) separates $Y_{t}$ from the past of the whole process $X_{t}^{-} ∖ P_{Y_{t}}$ , which is used in the algorithm to estimate the graph. (b) Process graph, which aggregates the information in the time series graph for better visualization (labels denote the lags).Reuse & Permissions
Figure 3
Iterative steps in the analysis of model Eq. (5), time series length $T = 1000$ , integration time step $d t = 0.01$ , sampling interval $Δ s = 100$ , MI and CMI estimated using $k = 100$ . The label ( $n . i$ ) indicates the iteration step. (0.0) shows the MI process graph. With an initial $n_{0} = 3$ the next step (3.0) (see the Supplemental Material [11]) with three conditions is already almost identical to the converged graph in step (3.1), where the values in the boxes denote the estimate $\hat{I^{DTE}}$ of TE via Eq. (3) with $τ^{*}$ chosen such that $I (X_{t - τ^{*} - 1}; Y_{t} | S_{Y_{t}, X_{t - τ^{*} - 1}})$ has declined below significance. Incorrect links and lag labels are in red.Reuse & Permissions
Figure 4
Analysis of time series of mean sea level pressure with $T = 1268$ using the same threshold as before and $τ_{\max } = 10$ . (0.0) depicts the process graph for MI. The iteration converges in step (2.1); step (2.0) and the lag functions are shown in the Supplemental Material [11]. Labels are as in Fig. 3.Reuse & Permissions

1 of 4

Sign up to receive regular email alerts from Physical Review Letters

This site uses cookies. To find out more, read our Privacy Policy. I Agree