Market structure explained by pairwise interactions

doi:10.1016/j.physa.2012.10.046

Physica A: Statistical Mechanics and its Applications

Volume 392, Issue 6, 15 March 2013, Pages 1375-1385

https://doi.org/10.1016/j.physa.2012.10.046 Get rights and content

Abstract

Financial markets are a typical example of complex systems where interactions between constituents lead to many remarkable features. Here we give empirical evidence, by making as few assumptions as possible, that the market microstructure capturing almost all of the available information in the data of stock markets does not involve higher order than pairwise interactions. We give an economic interpretation of this pairwise model. We show that it accurately recovers the empirical correlation coefficients; thus the collective behaviors are quantitatively described by models that capture the observed pairwise correlations but no higher-order interactions. Furthermore, we show that an order–disorder transition occurs, as predicted by the pairwise model. Last, we make the link with the graph-theoretic description of stock markets recovering the non-random and scale-free topology, shrinking length during crashes and meaningful clustering features, as expected.

Highlights

► No higher than pairwise interactions are involved in the interaction structure. ► Evidence of order–disorder is given and this is explained by the proposed model. ► The spectrum of the interaction matrix is related to the crashes. ► The remaining topology is, as expected, a scale-free structure, meaning a highly correlated state. ► It makes the link between, a priori, unconnected areas: finance, neurosciences and magnetic materials.

Keywords

Market microstructure

Pairwise interactions

Collective phenomena

Order–disorder

Financial network

1. Introduction

Complex systems are particularly interesting because they exhibit very sophisticated behaviors caused by, a priori, simple rules. Indeed, magnetic materials and neural networks, for instance, have some striking features such as phase transitions, memory, complicated equilibria structures and clustering. It is remarkable that these properties are caused by such simple interactions as pairwise ones. We believe that the markets are also driven by such simple rules and that the higher-order interactions encountered in financial systems are the pairwise ones. Typical characteristics of a complex system are numerous entities and interaction rules (with a degree of non-linearity), all leading to the emergence of collective behaviors. Those behaviors in general depend more on the interactions (e.g., their scaling and their order) and their effects than on the intrinsic nature of the elementary constitutive entities taken individually. The market can be viewed as such a system. The entities can be stocks or traders interacting through non-obvious rules. We note that we should interpret interaction at the larger sense of mutual or reciprocal influence.

What one knows is that the markets exhibit features such as synchronization [1], structural reorganization [2], [3], power laws [4], [5], hierarchical and non-randomness [6]. What one does not know is the true market dynamics. Even if trading rules are known, microscopic equations of motion are not known. This is a fundamental difference between finance and physics (or neuroscience).

A natural approach, given the above considerations, is a statistical modeling collecting and using at best the available amount of information and allowing (in a certain sense) the emergence of critical properties. This is exactly the purpose of the maximum entropy modeling in complex systems theory. Indeed the maximum entropy principle (MEP) allows the selection of the less restricting model on the basis of incomplete information [7]. We choose this data-based approach to avoid the use of any particular microscopic schemes (e.g. trader–agent-based rules, a priori unknown) which are difficult to assess experimentally or to avoid any analogy (even if some of such models are valuable [8]). The reason is that, even if one does not know the underlying microscopic processes, the macroscopic collective behaviors can still be described by an effective model. One has long experience of this powerful approach in the description of phase transitions and magnetic materials [9]. More recently, it has led to valuable results about the description of real neural networks [10]. Moreover, this approach also has counterparts in economics. Indeed, in addition of the statistical meaning of entropy, one can interpret it as a measure of economic activity [11] and it is linked to the central concept of utility of many interacting economic entities [12], [13].

An important outcome of such a modeling is a convenient simplified version of the real interaction structure that is still consistent with the data. In the following, we derive the model in this point of view and we study the structural properties of the resulting complex network. The critical properties will be investigated in another work.

The paper is organized as follow. In Section 2, we present the model, its economic interpretation and the link between the interaction matrix and the moments. In Section 3, we give evidence that the information embedded in the data is mostly explained by the pairwise but no higher-order interactions. In Section 4, we show an order–disorder transition through actual data. In Section 5, we highlight the properties of the interaction matrix and its link to the crises. Finally, in Section 6, we explain the link with the graph-theoretic approach and the topological evolution of the market network.

2. The model

2.1. Model derivation

The aim is to set up a statistical model describing the market state. This requires a way to infer the probability distribution in order to get the observables (here, the associated moments). The model will also allow the study of the market structure. All these quantities will be defined below. We consider a set of $N$ market indices or $N$ stocks with binary states $s_{i}$ ( $s_{i} = \pm 1$ for all $i = 1, \dots, N$ ). A system configuration will be described by a vector $s = (s_{1}, \dots, s_{N})$ . The binary variable will be equal to $1$ if the associated index is bullish and equal to $- 1$ if not. A configuration $(s_{1}, \dots, s_{N})$ is a binary version of the index returns. One knows that this approximation is already useful in the description of neural populations [10] and that neural networks are similar to financial networks [6]. We may think that it will also be the case in finance; this will be justified a posteriori as the model gives consistent results.

We seek to establish a less structured model explaining only the measured index mean orientation $q_{i} = 〈 s_{i} 〉$ and instantaneous pairwise correlations $q_{k l} = 〈 s_{k} s_{l} 〉$ . The brackets $〈 \cdot 〉$ denote the average with respect to the unknown distribution $p (s)$ . As the entropy of a distribution measures the randomness or the lack of interaction among the binary variables, a way to infer such probability distribution knowing the mean orientations and the correlations is the maximum entropy principle. Jaynes showed how to derive the probability distribution using the maximum entropy principle [14]; for supplementary information see Ref. [7]. It consists in the following constrained maximization: (1) $\begin{matrix} max & S (s) = - \sum_{{s}} p (s) log p (s) \\ s.t. & \sum_{{s}} p (s) = 1, & \sum_{{s}} p (s) s_{i} = q_{i}, & \sum_{{s}} p (s) s_{i} s_{j} = q_{i j} . \end{matrix}$

The resulting two-agent distribution $p_{2} (s)$ is the following: (2) $p_{2} (s) = Z^{- 1} exp (\frac{1}{2} \sum_{i, j}^{N} J_{i j} s_{i} s_{j} + \sum_{i = 1}^{N} h_{i} s_{i}) \equiv \frac{e^{- H (s)}}{Z},$ where $J_{i j}$ and $h_{i}$ are Lagrange multipliers and $Z$ a normalizing constant (the partition function). They can be expressed in terms of partial derivatives of the entropy as (3) $\frac{\partial S (s)}{\partial q_{i}} = - h_{i} \frac{\partial S (s)}{\partial q_{i j}} = - J_{i j} .$

Thus preferences are conjugated to mean orientations and interaction strengths to pairwise correlations. Including higher-order correlations in constraints in (1) could bring more information and thus decrease the maximum entropy. We will show below that this will not be the case.

The Gibbs distribution (2) is similar to the one given by Brock and Durlauf in the discrete choice problem [12] and to the one in stochastic models in macroeconomics [11], and also to the Ising model used in description of magnetic materials and neural networks [9], [10]. It is also a special case of Markov random fields [15]. It is to be noted that the Gibbs distribution and Shannon entropy naturally arise from the stochastic modeling in economics; this is discussed in Ref. [11].

We obtain the parameters ${J_{i j}, h_{i}}$ by performing explicitly the maximization (1) so that the theoretical moments $〈 s_{i} 〉$ and $〈 s_{i} s_{j} 〉$ match the measured ones $q_{i}$ and $q_{i j}$ . We note that this requires the computation of $2^{N}$ terms. If this number is large, the computation will take a while, and we can benefit from one of the methods described in Ref. [16].

Last, we show how the cumulants are obtained from this model and their relation to interaction strengths. As the statistical model (2) is expressed as a Gibbs distribution, we have the relations (4) ${〈 s_{i_{1}} \dots s_{i_{N}} 〉}_{c} = \partial^{N} ln Z / \partial h_{i_{1}} \dots \partial h_{i_{N}},$ where ${〈 \cdot 〉}_{c}$ is the cumulant average [17]; it gives the relation between $J$ and the correlation functions. If the partition function $Z$ cannot be explicitly computed, we can use the Plefka series [18] or a variational cumulant expansion [19].

Hereafter, we will show that the covariances are consistently deduced from this statistical model and thus that they are a function of the interaction strengths.

2.2. Interpretation

We interpret the objective function $H (s)$ defined by the MEP in the distribution (2) as follows. The pairwise interactions between economic agents are modeled by interaction strengths $J_{i j}$ which describe how $i$ and $j$ influence each other. Here, by interaction, we mean a measure of mutual influence or a measure of share comovement. In this framework, our intention is not to give a description of these interactions but to study their effects. Actually, the causes underlying the interaction process seem to be unnecessary in the description of emergent macroscopic behaviors. Indeed the complicated interactions between magnetic moments or between neurons are efficiently simplified in their maximum entropy description but one still recovers the main macroscopic features observed in these systems. In this description, the crucial features are the scaling (dependence on or independence of the system size) of interaction strengths and the order of interactions. The matrix $J$ is set to be symmetric in this first approach. There is disagreement or conflict between entities when the weighted product of their orientations $J_{i j} s_{i} s_{j}$ is negative. If two shares are supposed to move together ( $J_{i j} > 0$ ), a conflicting situation is the one where they do not have the same orientation (bearish or bullish).

We include the idiosyncratic preferences of the economic agents, here the willingness to be bullish or not. These Lagrange multipliers $h_{i}$ can also be interpreted as the external influences on entities $i$ induced by the macroeconomic background. By example a company can prosper and make benefits during a crisis period and the associated stock can still fall simultaneously because the investors are negatively influenced by the economic background. It results that the stock will have a propensity to fall. We denote the external influence by $h_{i}$ . If $i$ ’s orientation satisfies its preference, $h_{i} s_{i}$ is positive. The total conflict of the system is thus given by (5) $H (s) = - \frac{1}{2} \sum_{i = 1}^{N} \sum_{j = 1}^{N} J_{i j} s_{i} s_{j} - \sum_{i = 1}^{N} h_{i} s_{i} .$

So, we interpret $H (s)$ as the opposite of the so-called utility function $U (s) = - H (s)$ with pairwise interacting and idiosyncratic parts. Consequently the interaction strengths can be viewed as the incentive complementarities [12], [13]. Indeed we have $\partial^{2} U / \partial s_{i} \partial s_{j} = J_{i j}$ . The larger $J_{i j} s_{i} s_{j}$ is, the stronger the strategic interaction between $i$ and $j$ is.

We emphasize that this Ising like model is forced upon us as the statistically consistent model with the measured orientations and correlations. It is not an analogy based on specific hypotheses about the market dynamics.

3. Consistency of the pairwise modeling

One of the most exciting features of the model is the emergence of collective behaviors even if the interactions are weak. If the model is able to explain the recorded data, the system is therefore dominated by pairwise correlations. The aim is to provide quantitative empirical evidence that the pairwise modeling is a consistent paradigm to explain the financial data and exhibited behaviors in the market. In the following, we apply the pairwise model to a set of six major market indices (AEX, Bel-20, CAC 40, Xetra Dax, Eurostoxx 50, FTSE 100). We selected only European indices because some financial issues are specific to Europe and we consider indices because they are the driving force of the respective stock markets [20]; they will reflect the main properties of the subjacent stock set. We will say that they are up or bullish if the closing price is higher than the opening price and they are down or bearish if not. These will constitute our binary states. We observe 2253 configurations from 6/06/2002 to 14/06/2011 [21]. We take a nine year long time series including two large crises. The daily sampling is enough since we want to study large crises, and the two principal peaks of the Fourier transform are centered on frequencies $f_{1} = 6 \times 1 0^{- 4} d^{- 1}$ and $f_{2} = 1.2 \times 1 0^{- 3} d^{- 1}$ ; the unit day stands for trading day. The first frequency $f_{1}$ is the crisis occurrence frequency in our time window; the corresponding period is $T_{1} = 1.7 \times 10^{3} d$ . Later, we will also analyze the stocks composing the Dow Jones and the S&P100 indices, and another set of 116 stocks. First of all, we give the magnitude order of the interaction strengths and of the empirical pairwise correlations in Fig. 1.

The $J_{i j}$ are all positive; we can therefore use net mean orientation (net magnetization) as an order parameter. The mean value of $h_{i}$ is about 0.0113.

As mentioned above, higher-order interactions can be involved in the interaction structure. In order to show that pairwise correlations are prevailing, we compute the Kullback–Leibler (KL) divergence, $D_{KL} (P_{2} ∥ P_{data})$ between the two-agent maximum entropy (ME) distribution $P_{2}$ and the empirical one $P_{data}$ . The KL divergence is equal to $2.27 \times 1 0^{- 2}$ for the ME distribution inferred from 2253 observations. It must be compared to $D_{KL} (P_{1} ∥ P_{data}) = 1.4801$ for the independent agents model $P_{1}$ . The closer to zero this quantity is, the closer $P_{2}$ to $P_{data}$ is. Specifically, a consistent way to test if the pairwise correlation model satisfactorily explains the data statistics is to evaluate the ratio between $S (P_{1}) - S (P_{2})$ and the Kullback–Leibler discrepancy $I_{N} \equiv D_{KL} (P_{N} ∥ P_{1})$ , where $S (P_{2})$ is the entropy of the pairwise model. If this ratio is close to 1, the pairwise correlations explain most of the available information. Indeed the multi-information $I_{N} = S (P_{1}) - S (P_{N})$ measures the total amount of correlations in the system [22]. In this application, we obtain $I_{2} / I_{N} = 98.5 %$ . The pairwise correlation model is effective since it explains almost all the available information; only 1.5% of information is due to higher-order interactions.

As a further test of the pairwise model consistency, we show below that this statistical model is able to recover the observed empirical moments. We compare the average index orientations $q_{i} = T^{- 1} \sum_{t = 1}^{T} s_{i, t}$ obtained by simulation to the real ones. We simulated the process by doing 1×10⁵ equilibration Monte Carlo time steps (MCS) and we take the average on the next 2×10⁷ MCS in order to reduce the variance of the estimator. The flipping attempts are simulated by the Glauber dynamics. Namely, we take an entity $i$ chosen randomly and the attempt to flip the associated binary variable $s_{i}$ is performed with a rate depending on the exponential weight, the other orientations remaining fixed [23]. We take the time average for each index from the data and we compare it to the value obtained with the simulation; they are illustrated in Fig. 2.

The root mean squared error (RMSE) is equal to 7×10⁻⁴, which represents 1.5% of the root mean squared (RMS) value of the six arithmetic means (equal to $4.90 \times 1 0^{- 2}$ ). We recover quantitatively the average orientation of the six indices on the observation period. Moreover, since we obtained the probability distribution, we can compare the correlation coefficients resulting from the sampling of the proposed probability distribution to the empirical ones. We sample the probability law $p_{2} (s, J^{ME}, h^{ME})$ by a Monte Carlo Markov chain (MCMC). We take $1.2 \times 10^{6}$ equilibration steps and $1.2 \times 10^{4}$ independent sampling steps between each sample. Fig. 3 illustrates the recovered correlation coefficients with the maximum entropy estimation versus the empirical ones. The results for only 130 observations (chosen arbitrarily corresponding to half a year) are conclusive. Indeed the RMSE represents 8.3% of the RMS value and the correlation coefficient of the empirical and simulated values is equal to 0.963. Including more observations (2258 trading days) allows us to reduce the dispersion in the results (correlation coefficient of the empirical and simulated values equal to 0.997; the RMSE represents 1.8% of the RMS value). We note that it is effective even with few data.

We perform the same work for the Dow Jones and the S&P100 indices (2500 configurations observed from 10/10/2001 to 02/08/2011). We also consider 116 stocks from the New York Stock Exchange available on the Onnela website (http://jponnela.com/) extending from the beginning of 1982 to the end of 2000 (4800 trading days). For these larger stock sets, the exact entropy maximization (1) is not computationally tractable. There are several approximate inversion methods to estimate the parameters. The mean field methods (naïve, TAP and Tanaka’s inversion; see Refs. [16], [24], [25]) are the faster ones and they are accurate if the interaction strengths are weak (the weakness will be investigated in a further work). These methods will be used in the investigation of the structure evolution due to their reasonable accuracy and quickness. Two other valuable inference methods are minimum probability flow (MPF) [26] and regularized pseudo-likelihood maximization (rPLM) [27]. In our application the rPLM method performs best. The results for the first and second recovered moments (2×10⁶ equilibration MCS, values estimated on 2×10⁷ samples recorded each $N$ MCS) are illustrated in Fig. 4, Fig. 5.

The correlation coefficient between the recovered and empirical values is respectively 0.998, 0.996 and 0.997 for the net orientations illustrated in Fig. 4 and 0.989, 0.964, 0.997 for the covariances illustrated in Fig. 5, which shows the strong linear statistical relation between the empirical and the recovered values. The relative deviation between the RMSE and the RMS values is respectively 2%, 7% and 6% for the net orientations and 9%, 17%, 8% for the covariances.

We have seen that, in addition of the multi-information criterion, the net orientations and the covariances are recovered from this model even with few data. We conclude that the proposed pairwise interaction structure is a trustful one; this means that interactions are believed to be pairwise and symmetric ones and that they cause correlations.

4. Order–disorder transition

As the previous pairwise model describes market indices quantitatively, we expect to observe an order–disorder transition in this system; we give below some empirical evidence that these transitions actually appear. As the interaction strengths are all positive, the system is ordered if the net orientation distribution has two modes near the extreme values −1 and 1 and disordered if the distribution has a unique mode. Indeed in an ordered situation, each index tends to have the same orientation as the others. Furthermore, in the absence of external influences, both extreme values are equivalent (as a consequence of the symmetry under sign exchange), and the distribution is thus bimodal. One of the extreme values can be favored following the values taken by the external influences $h_{i}$ . It will be a first clue that the system is reorganized if the distribution changes in such a way (having two modes and then a unique one, and reciprocally). We compute the system net orientation $q (τ, Δ t) = {(Δ t N)}^{- 1} \sum_{i} \sum_{t = τ}^{τ + Δ t} s_{i, t}$ on successive periods $Δ t$ of 25 trading days (without overlapping), and we show that the net orientation probability distribution can be bimodal or not on successive time windows. The resulting empirical distributions for observations from 5 November 2010 to 30 March 2011 are illustrated in Fig. 6.

In Fig. 6 we see that the empirical probability distribution has initially two modes at extreme orientation values, then has no clear mode, and finally again has two modes. During this period, initially the indices move in an organized way then in a disorganized fashion, and finally the Fukushima nuclear accident caused a large global market fall followed by a large recovery. During this event, the indices were in comovement. So the system is initially ordered then disordered for two periods and then again ordered.

Another way to characterize financial irregularities is to study the entropy $S (s)$ on a sliding window (here, 300 trading days shifted by 1 day). We compute the mean-field approximation of the entropy on those time windows (much faster than the exact computation). The mean-field entropy [28] is (6) $S_{MF} (s) = - \sum_{i = 1}^{N} \frac{1 + q_{i}}{2} ln (\frac{1 + q_{i}}{2}) + \frac{1 - q_{i}}{2} ln (\frac{1 - q_{i}}{2}) .$

The entropy is maximal when the average orientations, computed on the corresponding time window, are equal to zero and is minimal when the indices have the same orientation. During a disordered period, the entropy should be large, and during a synchronized (ordered) period the entropy should be low. We should thus observe entropy minima simultaneously to orientation extrema (bubbles or crashes). We check in the results illustrated in Fig. 7 that orientation extrema and entropy minima are related to the periods of synchronization described in Ref. [1].

We observe large falls of the entropy when the net orientation is much larger than its mean (the mean is set to zero in Fig. 7). The shaded portions show the orientation extrema and entropy minima on this time window. They correspond (chronologically) to the end of the growth period and the end of the collapse. Furthermore the correlation coefficient of the net-orientation and the financial time series is equal to 0.82 showing a high degree of linear statistical dependency. We conclude that the entropy minima are thus related to financial irregularities (large upward or downward movements).

This is empirical evidence that order–disorder transitions occur in markets. This interpretation is supported by the recent results obtained in Ref. [1], where the authors showed that market irregularities present a high degree of synchronization, meaning an ordered state. The economic consequence is that the whole market is correlated when such transitions occur. It also means the absence of a characteristic scale for the fluctuations and the emergence of power-laws.

In Appendix, we illustrate in Fig. 14 a larger version of Fig. 7.

5. Dynamics of interactions

Linked to the above, such a transition occurs if the stochasticity changes or the interaction strengths change. A possible interpretation of time-varying interaction strengths is that some learning or adaptive process takes place through time. This means that the market adjusts the interactions between its entities in some adaptive processes so the ${J_{i j}, h_{i}}$ are time dependent. The reason is that the background, namely worldwide economic conditions, changes through time and goes through economic fluctuations with contractions (recessions) and expansions (growths). As the correlations are explained by the pairwise interactions, it also means that the correlations do not necessarily match past correlations.

Following this interpretation, we expect that the temporal behaviors of the interaction strengths and external influences are related to market evolution. This is indeed true, as we will see below. First of all, we study the preference evolution of the six previous indices (reflecting the current state of the European economy) and its link to the crises. We use a sliding temporal window of width $T = 200$ trading days shifted by a constant amount of $Δ t = 2$ trading days. We show that the aggregate preference $h = \sum_{i} h_{i}$ is negative during a crisis (or during a significative contraction), as illustrated in Fig. 8.

The first negative incursion corresponds to the 2002–2003 crisis and the second one to the 2008–2009 crisis [21]. As expected, the external influences are decreasing when the market is subject to a crisis.

More interestingly, we will study the spectrum of the interaction matrix. Indeed the spectrum evolution will be related to the market evolution. The spectrum of the interaction matrix of a stock set has an interesting feature; we will show it for the Dow Jones index. We collected data for the Dow Jones index from 10 October 2001 to 1 August 2011 [21], and we extract the interaction strengths using the third-order approximation described in Ref. [25]. The trace of the interaction matrix, the sum of its eigenvalues, has the following interesting property. It decreases during a crisis; specifically, the trace minus its temporal average becomes negative if there is a substantial fall of the index. This feature is illustrated in Fig. 9.

The trace of the exact interaction matrix should be zero (without self-interactions) but, with Tanaka’s diagonal trick, the diagonal entries are related to the second-order term and to a part of the third order of the Plefka series [25], [18]. The second-order term of the Plefka series is negative; the sign of the third-order term depends on the product of the interaction strengths. The temporal variation of the trace reflects the temporal variation of these second-order and third-order terms. These terms are particularly important near a transition. This explains why the trace of the interaction matrix is smaller than its mean value during a crisis. Indeed during a crisis all the stocks act in similar way: they fall down. They thus have similar mean orientation (down) and the resulting system state is an ordered one. Before the crisis, during a common market growth or steady state, the price of some stocks rises (on average) and some others fall, leading to a dispersion of the mean orientations. This is indirect evidence of a transition from one regime to another and of coordination. It is consistent with the results obtained above and in Refs. [1], [29]. In Appendix, we illustrate in Fig. 13 a larger version of the Fig. 9.

6. Link to the graph-theoretic approach

Hereafter, we make the link with the previous spectrum feature and the observation that the length of the minimum spanning tree (MST) based on the Sornette–Mantegna distance decreases during a crash [30], [31], meaning that stocks are highly correlated during these events (as they should be in an order–disorder transition). We will see that we recover this feature with the pairwise model with a distance based on interaction strengths in place of correlation coefficients. Indeed the interaction matrix can be thought of as the weight matrix of an undirected complete graph. Using a modified version of the method proposed in Ref. [32] and computing the minimum spanning tree length $L (t)$ (the sum of the edges weights of the MST), we also observe that this length decreases during a crash, as expected; the results for the Dow Jones index are illustrated in Fig. 10.

Moreover, it also allows cluster identification. Indeed, it is known that the asset tree based on the Sornette–Mantegna distance allows regrouping some stocks in clusters following their economic sectors [31]. As the correlations are caused by the interactions, it is not surprising that the MST of the network defined by the interaction matrix also allows cluster identification. This approach has the advantage of not being limited to linear or monotonic statistical dependencies. The clustering feature is illustrated in Fig. 11.

We note that General Electric (GE) is not the most connected node but it is a central one in the sense that it appears in three different clusters; as such it is still considered as the root of the MST and defines the generational direction. This approach provides a different classification as given in Ref. [31] or given by Forbes for instance. Indeed, the Forbes classification is given by sector then by industry. Disney and Walmart are classified in the same sector, services; this category is too vague to be an useful tag. Similarly, General Electric is tagged by Forbes as industrial goods and then as diversified machinery, but this company also provides financial services, aircraft engines, TV channel broadcasting, etc. It is then clear that this company should be classified with more than one tag, as does the proposed method. In this point of view, the internal structure of each company seems to be the crucial information to identify stock clusters.

We can also study the topological structure of the remaining asset tree during a crash and a growth period. We will see that, as expected, the degree distribution follows a power law. We consider the stocks of the S&P100 index on two intervals, from 1/10/2007 to 01/02/2009 (360 trading day crisis period) and from 1/02/2005 to 1/07/2007 (600 trading day growth period). The occurring frequencies of the vertex degrees are illustrated in Fig. 12.

This reveals that the degree distribution is a power law, $f (n) \sim n^{- α}$ , and the value of the exponent is similar for the both periods. For the growth period, we obtain $\hat{α} = 1.64 \pm 0.17$ , and during a crash $\hat{α} = 1.58 \pm 0.12$ . They can be included in the confidence interval of each other, so they are very similar. The maximum degree is $n = 8$ in the both periods. They are 58 vertices of degree $n = 1$ during the crash. This value is slightly larger (about 10%) than the one corresponding to the growth period, 52 vertices of degree $n = 1$ . This explains the difference between both exponents. The asset tree topology is thus slightly different during a crash. The main change is the variation of the interaction strengths (the graph weights) rather than the variation of the vertex degrees. In both regimes, the asset trees are thus scale-free networks. This implies that the edges are not drawn at random and the asset trees exhibit small-worldness, as observed with another method in Ref. [6]. Furthermore, the low value of this exponent implies that hubs (high-degree vertices) represent a significant part of the total number of vertices. The market is thus sensitive to the failure of a hub (a highly connected company) whereas the failure of a leaf (terminal node) will only slightly affect the market. By example the hypothetic failure of the American Express Company (AXP) would leave a fragmented market whereas the bankruptcy of Kraft Food Inc. (KFT) would not change the topology of the asset tree significantly; see Fig. 11. This could help in selecting the companies one has to save from an eventual bankruptcy in order to minimize the impact of such an event. This could also help in selecting which companies one has to monitor to prevent a hypothetical dramatic system failure.

7. Conclusion

We have seen that, without making assumptions on the market dynamics, the maximum entropy principle provides a rigorous pairwise model which is able to describe the data and the observed collective behaviors quantitatively. We showed that including higher-order interactions does not explain more than using the pairwise model, and thus that the collective phenomena emerge from simple pairwise interactions. To confirm this result, we showed that this statistical model is able to recover the empirical moments computed from the data, especially the mean orientations and the correlations. The success of the pairwise model implies that markets exhibit some properties observed in magnetic materials and in neural networks. Indeed, we showed that an order–disorder transition occurs in such a system, as described by a pairwise model equivalent to the Ising model. Furthermore, we showed that the interaction strengths are time dependent, meaning that an adaptive process occurs and that they are the starting point of the graph-theoretic approach of the market. In this view the system is more than the sum of its parts, is ruled by its entities pairs, exhibits collective behaviors and is quantitatively described by a pairwise model. It is surprising that such sophisticated collective behaviors, emergent structures and underlying complex trading rules are captured by a simple (a priori) scheme of interdependence involving only pairwise but no higher-order interactions.

Acknowledgments

I would like to thank B. De Rock and P. Emplit for their helpful comments and discussions. This work was undertaken with financial support from the Solvay Brussels School of Economics and Management.

Appendix.

See Fig. 13, Fig. 14.

References

[1]
T. Dal’Maso Peron, F. Rodrigues
Collective behavior in financial markets
EPL (Europhysics Letters), 96 (2011), p. 48004
CrossRef View in Scopus Google Scholar
[2]
J.-P. Onnela, A. Chakraborti, K. Kaski, J. Kertész, A. Kanto
Dynamics of market correlations: taxonomy and portfolio analysis
Physical Review E, 68 (2003), p. 056110
Google Scholar
[3]
T.K.D. Peron, L. da Fontoura Costa, F.A. Rodrigues
The structure and resilience of financial market networks
Chaos: An Interdisciplinary Journal of Nonlinear Science, 22 (2012), p. 013117
Google Scholar
[4]
H. Stanley, X. Gabaix, P. Gopikrishnan, V. Plerou
Economic fluctuations and statistical physics: quantifying extremely rare and less rare events in finance
Physica A: Statistical Mechanics and its Applications, 382 (2007), pp. 286-301
View PDF View article View in Scopus Google Scholar
[5]
H. Stanley, V. Plerou, X. Gabaix
A statistical physics view of financial fluctuations: evidence for scaling and universality
Physica A: Statistical Mechanics and its Applications, 387 (2008), pp. 3967-3981
View PDF View article View in Scopus Google Scholar
[6]
P. Vértes, R. Nicol, S. Chapman, N. Watkins, E. Bullmore
Frontiers: topological isomorphisms of human brain and financial market networks
Frontiers in Systems Neuroscience (2011)
Google Scholar
[7]
T. Cover, J. Thomas, J. Wiley, et al.
Elements of Information Theory, Vol. 6
Wiley Online Library (1991)
Google Scholar
[8]
B. Rosenow, P. Gopikrishnan, V. Plerou, H. Stanley
Random magnets and correlations of stock price fluctuations
Physica A, 314 (2002), pp. 762-767
View PDF View article View in Scopus Google Scholar
[9]
K.H. Fischer, J.A. Hertz
Spin Glasses
Cambridge University Press, Cambridge (1991)
Google Scholar
[10]
E. Schneidman, M.J. Berry, R. Segev, W. Bialek
Weak pairwise correlations imply strongly correlated network states in a neural population
Nature, 440 (2006), pp. 1007-1012
CrossRef View in Scopus Google Scholar
[11]
M. Aoki
New Approaches to Macroeconomic Modeling: Evolutionary Stochastic Dynamics, Multiple Equilibria, and Externalities as Field Effects
Cambridge Univ. Pr. (1998)
Google Scholar
[12]
W.A. Brock, S.N. Durlauf
Discrete choice with social interactions
Review of Economic Studies, 68 (2001), pp. 235-260
CrossRef View in Scopus Google Scholar
[13]
A. Mas-Colell, M. Whinston, J. Green, U.P.F.F. de Cičncies
Econňmiques i Empresarials, Microeconomic Theory, vol. 1, Oxford University Press, New York (1995)
Google Scholar
[14]
E.T. Jaynes
Information theory and statistical mechanics
Physical Review, 106 (1957), pp. 620-630
View in Scopus Google Scholar
[15]
R. Kindermann
Markov Random Fields and Their Applications (Contemporary Mathematics; V.1)
Amer Mathematical Society (1980)
Google Scholar
[16]
Y. Roudi, J. Tyrcha, J. Hertz
Ising model for neural data: model quality and approximate methods for extracting functional connectivity
Physical Review E, 79 (2009), p. 051915+
Google Scholar
[17]
R. Kubo
Generalized cumulant expansion method
Journal of the Physical Society of Japan, 17 (1962), p. 1100
View in Scopus Google Scholar
[18]
T. Plefka
Convergence condition of the tap equation for the infinite-ranged Ising spin glass model
Journal of Physics A: Mathematical and General, 15 (1982), p. 1971
CrossRef View in Scopus Google Scholar
[19]
D. Barber, P. van de Laar
Variational cumulant expansions for intractable distributions
Journal of Artificial Intelligence Research, 10 (1999), pp. 435-455
CrossRef View in Scopus Google Scholar
[20]
Y. Shapira, D.Y. Kenett, E. Ben-Jacob
The index cohesive effect on stock market correlations
The European Physical Journal B—Condensed Matter and Complex Systems (2009)
Google Scholar
[21]
Yahoo, 2011. http://finance.yahoo.com/.
Google Scholar
[22]
E. Schneidman, S. Still, M.J. Berry, W. Bialek
Network information and connected correlations
Physical Review Letters, 91 (2003), p. 238701+
Google Scholar
[23]
R.J. Glauber
Time-dependent statistics of the Ising model
Journal of Mathematical Physics, 4 (1963), pp. 294-307
CrossRef View in Scopus Google Scholar
[24]
M. Schaub, S. Schultz
The Ising decoder: reading out the activity of large neural ensembles
Journal of Computational Neuroscience (2010), pp. 1-18
View in Scopus Google Scholar
[25]
T. Tanaka
Mean-field theory of Boltzmann machine learning
Physical Review E, 58 (1998), pp. 2302-2310
View in Scopus Google Scholar
[26]
J. Sohl-Dickstein, P.B. Battaglino, M.R. DeWeese
New method for parameter estimation in probabilistic models: minimum probability flow
Physical Review Letters, 107 (2011), p. 220601
CrossRef View in Scopus Google Scholar
[27]
E. Aurell, M. Ekeberg
Inverse Ising inference using all the data
Physical Review Letters, 108 (2012), p. 090201
CrossRef View in Scopus Google Scholar
[28]
K. Binder, A. Young
Spin glasses: experimental facts, theoretical concepts, and open questions
Reviews of Modern physics, 58 (1986), p. 801
View in Scopus Google Scholar
[29]
L. Sandoval, I. Franca
Correlation of financial markets in times of crisis
Physica A Statistical Mechanics and its Applications, 391 (2012), pp. 187-208
View PDF View article View in Scopus Google Scholar
[30]
R. Mantegna
Hierarchical structure in financial markets
The European Physical Journal B—Condensed Matter and Complex Systems, 11 (1999), pp. 193-197
View in Scopus Google Scholar
[31]
J. Onnela, A. Chakraborti, K. Kaski, J. Kertesz
Dynamic asset trees and black Monday
Physica A: Statistical Mechanics and its Applications, 324 (2003), pp. 247-252
View PDF View article View in Scopus Google Scholar
[32]
C. Ding, X. He, H. Xiong, H. Peng
Transitive closure and metric inequality of weighted graphs: detecting protein interaction modules using cliques
International Journal of Data Mining and Bioinformatics, 1 (2006), pp. 162-177
CrossRef View in Scopus Google Scholar

Cited by (43)

Market basket analysis by solving the inverse Ising problem: Discovering pairwise interaction strengths among products
2019, Physica A: Statistical Mechanics and its Applications
Citation Excerpt :
In this line, we demonstrate that the Ising model approach is also feasible for describing the aggregate purchasing behavior of a large number of consumers. This approach allows an economic interpretation, in which the inferred parameters of the distribution can be understood as a measure of economic activity among the elements that constitute the system [20]. The results indicate that this kind of models are able to recover satisfactorily the distribution of the states.
Financial price dynamics and phase transitions in the stock markets
2023, European Physical Journal B
Temperature-dependent performance of the erasure machine
2022, Journal of the Korean Physical Society
Inference of Interactions between Players for Asynchronously Updated Evolutionary Game
2022, SSRN
Eye-tracking as a proxy for coherence and complexity of texts
2021, PLoS ONE
Inverse problems for structured datasets using parallel TAP equations and restricted Boltzmann machines
2021, Scientific Reports

View all citing articles on Scopus

View Abstract

[1] [1]
T. Dal’Maso Peron, F. Rodrigues
Collective behavior in financial markets
EPL (Europhysics Letters), 96 (2011), p. 48004
CrossRef View in Scopus Google Scholar

[2] [2]
J.-P. Onnela, A. Chakraborti, K. Kaski, J. Kertész, A. Kanto
Dynamics of market correlations: taxonomy and portfolio analysis
Physical Review E, 68 (2003), p. 056110
Google Scholar

[3] [3]
T.K.D. Peron, L. da Fontoura Costa, F.A. Rodrigues
The structure and resilience of financial market networks
Chaos: An Interdisciplinary Journal of Nonlinear Science, 22 (2012), p. 013117
Google Scholar

[4] [4]
H. Stanley, X. Gabaix, P. Gopikrishnan, V. Plerou
Economic fluctuations and statistical physics: quantifying extremely rare and less rare events in finance
Physica A: Statistical Mechanics and its Applications, 382 (2007), pp. 286-301
View PDF View article View in Scopus Google Scholar

[5] [5]
H. Stanley, V. Plerou, X. Gabaix
A statistical physics view of financial fluctuations: evidence for scaling and universality
Physica A: Statistical Mechanics and its Applications, 387 (2008), pp. 3967-3981
View PDF View article View in Scopus Google Scholar

[6] [6]
P. Vértes, R. Nicol, S. Chapman, N. Watkins, E. Bullmore
Frontiers: topological isomorphisms of human brain and financial market networks
Frontiers in Systems Neuroscience (2011)
Google Scholar

[7] [7]
T. Cover, J. Thomas, J. Wiley, et al.
Elements of Information Theory, Vol. 6
Wiley Online Library (1991)
Google Scholar

[8] [8]
B. Rosenow, P. Gopikrishnan, V. Plerou, H. Stanley
Random magnets and correlations of stock price fluctuations
Physica A, 314 (2002), pp. 762-767
View PDF View article View in Scopus Google Scholar

[9] [9]
K.H. Fischer, J.A. Hertz
Spin Glasses
Cambridge University Press, Cambridge (1991)
Google Scholar

[10] [10]
E. Schneidman, M.J. Berry, R. Segev, W. Bialek
Weak pairwise correlations imply strongly correlated network states in a neural population
Nature, 440 (2006), pp. 1007-1012
CrossRef View in Scopus Google Scholar

[11] [11]
M. Aoki
New Approaches to Macroeconomic Modeling: Evolutionary Stochastic Dynamics, Multiple Equilibria, and Externalities as Field Effects
Cambridge Univ. Pr. (1998)
Google Scholar

[12] [12]
W.A. Brock, S.N. Durlauf
Discrete choice with social interactions
Review of Economic Studies, 68 (2001), pp. 235-260
CrossRef View in Scopus Google Scholar

[13] [13]
A. Mas-Colell, M. Whinston, J. Green, U.P.F.F. de Cičncies
Econňmiques i Empresarials, Microeconomic Theory, vol. 1, Oxford University Press, New York (1995)
Google Scholar

[14] [14]
E.T. Jaynes
Information theory and statistical mechanics
Physical Review, 106 (1957), pp. 620-630
View in Scopus Google Scholar

[15] [15]
R. Kindermann
Markov Random Fields and Their Applications (Contemporary Mathematics; V.1)
Amer Mathematical Society (1980)
Google Scholar

[16] [16]
Y. Roudi, J. Tyrcha, J. Hertz
Ising model for neural data: model quality and approximate methods for extracting functional connectivity
Physical Review E, 79 (2009), p. 051915+
Google Scholar

[17] [17]
R. Kubo
Generalized cumulant expansion method
Journal of the Physical Society of Japan, 17 (1962), p. 1100
View in Scopus Google Scholar

[18] [18]
T. Plefka
Convergence condition of the tap equation for the infinite-ranged Ising spin glass model
Journal of Physics A: Mathematical and General, 15 (1982), p. 1971
CrossRef View in Scopus Google Scholar

[19] [19]
D. Barber, P. van de Laar
Variational cumulant expansions for intractable distributions
Journal of Artificial Intelligence Research, 10 (1999), pp. 435-455
CrossRef View in Scopus Google Scholar

[20] [20]
Y. Shapira, D.Y. Kenett, E. Ben-Jacob
The index cohesive effect on stock market correlations
The European Physical Journal B—Condensed Matter and Complex Systems (2009)
Google Scholar

[21] [21]
Yahoo, 2011. http://finance.yahoo.com/.
Google Scholar

[22] [22]
E. Schneidman, S. Still, M.J. Berry, W. Bialek
Network information and connected correlations
Physical Review Letters, 91 (2003), p. 238701+
Google Scholar

[23] [23]
R.J. Glauber
Time-dependent statistics of the Ising model
Journal of Mathematical Physics, 4 (1963), pp. 294-307
CrossRef View in Scopus Google Scholar

[24] [24]
M. Schaub, S. Schultz
The Ising decoder: reading out the activity of large neural ensembles
Journal of Computational Neuroscience (2010), pp. 1-18
View in Scopus Google Scholar

[25] [25]
T. Tanaka
Mean-field theory of Boltzmann machine learning
Physical Review E, 58 (1998), pp. 2302-2310
View in Scopus Google Scholar

[26] [26]
J. Sohl-Dickstein, P.B. Battaglino, M.R. DeWeese
New method for parameter estimation in probabilistic models: minimum probability flow
Physical Review Letters, 107 (2011), p. 220601
CrossRef View in Scopus Google Scholar

[27] [27]
E. Aurell, M. Ekeberg
Inverse Ising inference using all the data
Physical Review Letters, 108 (2012), p. 090201
CrossRef View in Scopus Google Scholar

[28] [28]
K. Binder, A. Young
Spin glasses: experimental facts, theoretical concepts, and open questions
Reviews of Modern physics, 58 (1986), p. 801
View in Scopus Google Scholar

[29] [29]
L. Sandoval, I. Franca
Correlation of financial markets in times of crisis
Physica A Statistical Mechanics and its Applications, 391 (2012), pp. 187-208
View PDF View article View in Scopus Google Scholar

[30] [30]
R. Mantegna
Hierarchical structure in financial markets
The European Physical Journal B—Condensed Matter and Complex Systems, 11 (1999), pp. 193-197
View in Scopus Google Scholar

[31] [31]
J. Onnela, A. Chakraborti, K. Kaski, J. Kertesz
Dynamic asset trees and black Monday
Physica A: Statistical Mechanics and its Applications, 324 (2003), pp. 247-252
View PDF View article View in Scopus Google Scholar

[32] [32]
C. Ding, X. He, H. Xiong, H. Peng
Transitive closure and metric inequality of weighted graphs: detecting protein interaction modules using cliques
International Journal of Data Mining and Bioinformatics, 1 (2006), pp. 162-177
CrossRef View in Scopus Google Scholar

Outline

Cited By (43)

Figures (14)

Physica A: Statistical Mechanics and its Applications

Abstract

Highlights

Keywords

1. Introduction

2. The model

2.1. Model derivation

2.2. Interpretation

3. Consistency of the pairwise modeling

4. Order–disorder transition

5. Dynamics of interactions

6. Link to the graph-theoretic approach

7. Conclusion

Acknowledgments

Appendix.

References

Cited by (43)

Market basket analysis by solving the inverse Ising problem: Discovering pairwise interaction strengths among products

Financial price dynamics and phase transitions in the stock markets

Temperature-dependent performance of the erasure machine

Inference of Interactions between Players for Asynchronously Updated Evolutionary Game

Eye-tracking as a proxy for coherence and complexity of texts

Inverse problems for structured datasets using parallel TAP equations and restricted Boltzmann machines

Extended string field theory for massless higher-spin fields

Bronchoscopical Repair of Tracheal Laceration Using Conventional Instrumentation: A Novel Technique

A global approach for learning sparse Ising models