Free energy and independent spins
These notes continue Lecture 1. Numbered references take you to the corresponding result. Links marked extra open an explanation; use “Back to the text” to resume reading.
Spin configurations
In Theorem 1.4, the maximum-entropy principle led to the Gibbs weight \(e^{-\beta H}\). We now apply that framework to a system of \(N\in\mathbb{N}\) two-valued spins. All logarithms are natural.
For \(N\in\mathbb N\), let \(\Omega_N=\{-1,+1\}^N\) and \(\mathcal F_N=\mathcal P(\Omega_N)\). A configuration \(\sigma=(\sigma_1,\ldots,\sigma_N)\in\Omega_N\) specifies the values of all \(N\) spins. The coordinate observables are \(S_i:\Omega_N\to\{-1,+1\}\), \(S_i(\sigma)=\sigma_i\).
The total magnetization and the empirical magnetization per spin are, respectively,
There are \(2^N\) microstates, but only \(N+1\) possible magnetizations. Different microstates can have the same magnetization: for example, \((+1,+1,-1,-1)\) and \((+1,-1,+1,-1)\) both have \(\widehat m_4=0\). The observable \(\widehat m_N\) records a macroscopic feature of a configuration without retaining all its microscopic information.
Let \(H_N:\Omega_N\to\mathbb R\) be a fixed interaction term, independent of the parameters. For two parameters \((\beta,h)\in\mathbb R^2\), we define
The term \(-\beta H_N(\sigma)\) is the interaction contribution, while \(hM_N(\sigma)\) is the contribution of the magnetization coupled to the external field \(h\).
The corresponding Gibbs measure on \((\Omega_N,\mathcal F_N)\) is
To connect with the convention of Lecture 1, write \(B\) for the external field and \(H_{N,B}(\sigma):=H_N(\sigma)-BM_N(\sigma)\) for the energy. The Gibbs weight is then
For \(\beta>0\), setting \(h:=\beta B\) gives the Hamiltonian and Gibbs weight in Definition 2.2. From now on, however, we treat \(\beta\) and \(h\) as independent mathematical parameters.
The finite sums also make sense for every real \(\beta\); positive temperature corresponds to \(\beta>0\).
As in Definition 1.1, observables become random variables once the probability measure has been specified. We abbreviate expectation under \(\mu_{N;\beta,h}\) by \(\mathbb E_{N;\beta,h}\). For \(A,C:\Omega_N\to\mathbb R\),
Pressure and free energy
The pressure per spin is \(p_N(\beta,h):=N^{-1}\log Z_N(\beta,h)\). For \(\beta>0\), the Helmholtz free energy per spin is
The mean magnetization per spin and the susceptibility with respect to the external field are
The following thermodynamic identities hold.
Fix \(N\) and a parameter-independent energy \(H_N\). For every finite \((\beta,h)\in\mathbb R^2\),
More generally, if the observable \(A:\Omega_N\to\mathbb R\) does not depend on \(\beta\) or \(h\), then
All sums are finite, so we may differentiate them term by term. In this proof only, write \(\mu=\mu_{N;\beta,h}\), \(\mathbb E=\mathbb E_{N;\beta,h}\), and \(Z=Z_N(\beta,h)\). Directly from Definition 2.2,
Dividing by \(NZ\) gives \(\partial_\beta p_N=-N^{-1}\mathbb E[H_N]\) and \(\partial_h p_N=N^{-1}\mathbb E[M_N]\).
Next differentiate the logarithm of a Gibbs mass. Since every mass is positive,
Multiplying by \(\mu(\sigma)\) yields
For a parameter-independent observable \(A\), we can therefore differentiate its expectation:
Apply the first identity with \(A=H_N\) to the formula for \(\partial_\beta p_N\):
Apply the second identity with \(A=M_N\) to the formula for \(\partial_h p_N\):
Finally, differentiate \(\partial_\beta p_N=-N^{-1}\mathbb E[H_N]\) in \(h\):
Repeating the calculation in the opposite order, we differentiate \(\partial_h p_N=N^{-1}\mathbb E[M_N]\) in \(\beta\) and use the first covariance identity:
Thus the same expression is obtained in either order.
□Note that \(\operatorname{Var}_{N;\beta,h}(H_N)>0\) precisely when \(H_N\) is nonconstant. The total magnetization is always nonconstant for \(N\ge1\), so \(\chi_N(\beta,h)>0\). Thus \(h\mapsto m_N(\beta,h)\) is strictly increasing at finite volume. Also, \(\chi_N(\beta,h)=N\operatorname{Var}_{N;\beta,h}(\widehat m_N)\).
Convexity and analyticity
The nonnegative second derivatives in Proposition 2.4 show convexity in each parameter separately. The covariance identity gives the stronger statement of joint convexity.
The function \((\beta,h)\mapsto p_N(\beta,h)\) is convex on \(\mathbb R^2\). Explicitly, for \(\theta_1,\theta_2\in\mathbb R^2\) and \(\lambda\in[0,1]\),
By Proposition 2.4, the Hessian is
where all moments are under \(\mu_{N;\beta,h}\). For \(v=(a,b)^{\mathsf T}\in\mathbb R^2\),
Thus the Hessian is positive semidefinite. To conclude directly, restrict \(p_N\) to any line segment: \(q(t):=p_N(\theta_1+t(\theta_2-\theta_1))\) has \(q''(t)\ge0\), so \(q\) is convex on \([0,1]\). Its convexity inequality is exactly the claimed inequality for \(p_N\).
□For every fixed \(N\), the functions \(Z_N\), \(p_N\), \(m_N\) and \(\chi_N\) are real analytic on \(\mathbb R^2\). The free energy \(f_N\) is real analytic on \((0,\infty)\times\mathbb R\).
For each configuration \(\sigma\), the function \((\beta,h)\mapsto-\beta H_N(\sigma)+hM_N(\sigma)\) is affine. Its exponential is real analytic. Since \(Z_N\) is a finite sum of these exponentials, it is real analytic as well.
Moreover, \(Z_N(\beta,h)>0\) at every real pair \((\beta,h)\). The function \(\log x\) is real analytic for \(x>0\), so \(p_N=N^{-1}\log Z_N\) is real analytic. Derivatives of real analytic functions remain real analytic; the identities \(m_N=\partial_hp_N\) and \(\chi_N=\partial_h^2p_N\) from Proposition 2.4 give the remaining claims. Finally, \(f_N=-p_N/\beta\) is real analytic wherever \(\beta>0\).
□Can a singularity nevertheless appear when \(N\to\infty\)? Yes: analyticity need not survive even uniform convergence on the real axis.
For \(t\in\mathbb R\) and \(N\in\mathbb N\), set
Every \(g_N\) is real analytic, but \(g_N\to g\) uniformly on \(\mathbb R\), where \(g(t)=|t|\) is not differentiable at \(t=0\).
The function \(\cosh(Nt)\) is positive and real analytic on \(\mathbb R\), proving the first assertion. For every real \(t\),
Take logarithms and divide by \(N\):
The left and right derivatives of \(g\) at zero are \(-1\) and \(+1\), respectively. Therefore the uniform limit is not differentiable at zero.
□For a spin model, one studies the thermodynamic limit \(p(\beta,h):=\lim_{N\to\infty}p_N(\beta,h)\), when it exists. Proposition 2.6 alone does not ensure that this limit is analytic, or that derivatives can be passed through the limit. The example explains how finite-volume regularity can coexist with a singular limiting pressure.
The product law at \(\beta=0\)
We now set \(\beta=0\) while keeping \(h\in\mathbb R\) fixed. Then \(H_{N;0,h}(\sigma)=hM_N(\sigma)\), so the energy \(H_N\) no longer contributes to the Gibbs weight. The effect of the field remains.
Under \(\mu_{N;0,h}\), the coordinates \(S_1,\ldots,S_N\) are independent and identically distributed, with common law
In particular,
Each spin has mean and variance
Here \(\cosh h=(e^h+e^{-h})/2\) and \(\tanh h=(e^h-e^{-h})/(e^h+e^{-h})\).
Since \(M_N(\sigma)=\sum_{i=1}^N\sigma_i\), the weight factorizes:
Summing separately over each coordinate gives
Dividing the factorized weight by this normalizing constant,
This product of one-spin probabilities is exactly independence, with the same marginal law at every site. Finally,
Because \(S_i^2=1\), its variance is \(1-(\tanh h)^2\). The identity \(\cosh^2h-\sinh^2h=1\), where \(\sinh h=(e^h-e^{-h})/2\), gives \(1-\tanh^2h=1/\cosh^2h\).
□As a check on the thermodynamic identities, Proposition 2.8 immediately gives
From now on, fix \(h\in\mathbb R\) and abbreviate the one-spin mean, variance and standard deviation by
Independence also gives \(\mathbb E_{N;0,h}[\widehat m_N]=m_h\) and \(\operatorname{Var}_{N;0,h}(\widehat m_N)=v_h/N\). We can now describe the behavior of this random empirical magnetization as the number of spins grows.
Weak and strong laws of large numbers
First recall the probabilistic theorem. It concerns an infinite sequence of random variables on a single probability space. We will then state its consequences using the Gibbs measures just constructed.
Let \((X_i)_{i\ge1}\) be independent and identically distributed real random variables on \((\Omega,\mathcal F,\mathbb P)\), with \(\mathbb E[X_1^2]<\infty\). Put \(m=\mathbb E[X_1]\), \(s^2=\operatorname{Var}(X_1)\), and \(\overline X_N=N^{-1}\sum_{i=1}^NX_i\). Then:
Weak law (WLLN). For every \(\varepsilon>0\),
Strong law (SLLN).
One infinite product space, with finite marginals
For each \(N\), \(\mu_{N;0,h}\) is a measure on \(\Omega_N=\{-1,+1\}^N\). These spaces change with \(N\). To discuss the limit of partial averages for one fixed configuration, we place all spins on the following common space.
Let
The product \(\sigma\)-algebra \(\mathcal F_\infty\) is generated by the coordinate maps \(S_i(\sigma)=\sigma_i\). Under \(\mu\), these maps are i.i.d. with law \(\rho_{0,h}\). Define the projection onto the first \(N\) coordinates by
For fixed \(h\), abbreviate \(\mu_N:=\mu_{N;0,h}\). Its relation to the infinite product measure is
To verify the last identity, first take \(A=\{(a_1,\ldots,a_N)\}\). The cylinder event \(\pi_N^{-1}(A)\) prescribes these \(N\) spin values and leaves every later spin unrestricted. Its product-measure probability is
Every subset of the finite set \(\Omega_N\) is a finite disjoint union of such singletons, so additivity proves the identity for all \(A\).
Lift the finite empirical magnetization to the common space by defining
This extra superscript only records the domain: \(\widehat m_N\) acts on \(N\)-tuples, whereas \(\widehat m_N^\infty\) acts on infinite configurations and reads their first \(N\) entries.
Fix \(h\in\mathbb R\). In the zero-interaction model:
Finite-volume weak law. For every \(\varepsilon>0\),
Strong law on the infinite product space.
Here \(m_h=\tanh h\) and \(v_h=1-\tanh^2h\).
By Proposition 2.8, the spins are i.i.d. with mean \(m_h\) and variance \(v_h\). On the finite space, \(\operatorname{Var}_{N;0,h}(\widehat m_N)=v_h/N\), so Chebyshev's inequality gives the displayed weak-law bound. On \((\Omega_\infty,\mathcal F_\infty,\mu_{0,h})\), apply the SLLN in Theorem 2.9 to the coordinate sequence \((S_i)_{i\ge1}\). Its partial averages are precisely \(\widehat m_N^\infty\), proving the strong law.
□The finite-volume WLLN is equivalent to its formulation on the common space, because for each \(N\),
The SLLN says that, for almost every infinite spin configuration and every tolerance \(\varepsilon>0\), all sufficiently long partial averages stay within \(\varepsilon\) of \(m_h\). The WLLN only states that the probability of violating that tolerance tends to zero.
The central limit theorem at \(\beta=0\)
The laws of large numbers identify the limiting magnetization. The central limit theorem describes the size and the distribution of the fluctuations around that value.
Let \((X_i)_{i\ge1}\) be i.i.d. real random variables with mean \(m\) and variance \(0<s^2<\infty\). Let \(s\) be their positive standard deviation and \(\overline X_N=N^{-1}\sum_{i=1}^NX_i\). Then
The arrow denotes convergence in distribution. The standard normal distribution has density \(y\mapsto(2\pi)^{-1/2}e^{-y^2/2}\).
This is the standard i.i.d. CLT. The assumptions are automatically satisfied by the spins of Proposition 2.8: for every fixed finite \(h\), their variance is \(v_h=1/\cosh^2h>0\).
Fix \(h\in\mathbb R\). Define the standardized magnetization on \((\Omega_N,\mathcal F_N,\mu_{N;0,h})\) by
Then, as \(N\to\infty\),
Equivalently, for every fixed \(x\in\mathbb R\),
On the infinite product space from Definition 2.10, apply Theorem 2.12 with \(X_i=S_i\), \(m=m_h\), and \(s=s_h\). This gives \(Y_N\circ\pi_N\Longrightarrow\mathcal N(0,1)\) under \(\mu_{0,h}\). Its distribution equals the distribution of \(Y_N\) under \(\mu_{N;0,h}\), by the marginal identity. This proves the first limit on the finite spaces.
Multiplying by the fixed positive constant \(s_h\) gives the second limit, since \(s_hG\) is normal with mean zero and variance \(s_h^2=v_h\). Finally, the normal distribution function is continuous at every \(x\), so convergence in distribution gives the stated convergence of distribution functions.
□Sources
S. Friedli and Y. Velenik, Statistical Mechanics of Lattice Systems. §1.1.5.
For \(N=1\), the configuration space is
The total magnetization is
Therefore:
- If \(\sigma=+1\), then \(M_1(\sigma)=+1\).
- If \(\sigma=-1\), then \(M_1(\sigma)=-1\).
Convergence in distribution concerns the laws of random variables, so the variables need not be defined on the same probability space. In Corollary 2.13, fix \(h\), write \(\mu_N:=\mu_{N;0,h}\), and let \(Y_N\) be the standardized magnetization defined there. Its law on \(\mathbb R\) is \(\nu_N(A):=\mu_N(Y_N\in A)\), for every Borel set \(A\subseteq\mathbb R\). We say that \(Y_N\) converges in distribution to a standard normal random variable when these laws converge to \(\mathcal N(0,1)\).
Two equivalent formulations. This convergence can be expressed in either of the following ways.
Through probabilities: the distribution functions satisfy
For a general limiting distribution function \(F\), convergence is required at every continuity point of \(F\). Here \(\Phi\) is continuous everywhere.
Through characteristic functions: with \(i^2=-1\),
The expectation defining a characteristic function always exists, since \(|e^{itY_N}|=1\). The equivalence with convergence in distribution is the characteristic-function criterion; the implication from characteristic functions follows from Lévy's continuity theorem. Friedli–Velenik, Theorem B.47.
Under \(\mu_N\) or under \(\mu\)? The corollary is stated for \(Y_N\) under the finite-volume law \(\mu_N\). On the infinite product space of Definition 2.10, put \(\mu:=\mu_{0,h}\) and \(Y_N^\infty:=Y_N\circ\pi_N\). The marginal relation \(\mu_N=\mu\circ\pi_N^{-1}\) gives
Thus \(Y_N\) under \(\mu_N\) and \(Y_N^\infty\) under \(\mu\) have exactly the same law for every \(N\). Their distribution functions and characteristic functions coincide, so the two CLT formulations are equivalent. Unlike almost-sure convergence in the SLLN, convergence in distribution does not require a common underlying probability space.
The thermodynamic relation is \(F=U-TS\): free energy equals internal energy minus temperature times entropy. Here is how it follows from the Gibbs measure, with all normalizations explicit.
Use units in which Boltzmann's constant is \(k_B=1\). For \(\beta>0\), set \(T=1/\beta\) and \(B=h/\beta\). The full physical energy is \(H_{N,B}=H_N-BM_N\). Define
Using the Shannon entropy from Definition 1.2 and \(\log\mu_{N;\beta,h}(\sigma)=-\beta H_{N,B}(\sigma)-\log Z_N(\beta,h)\),
Consequently,
Dividing by \(N\) gives the free energy per spin in Definition 2.3.
Our partition function is \(Z_N(\beta,h)=\sum_\sigma e^{-\beta H_N(\sigma)+hM_N(\sigma)}\), with \(h\) independent of \(\beta\). Therefore
To keep the external field \(B\) fixed, instead follow the curve \(h=\beta B\). The chain rule and Proposition 2.4 give
This is the derivative that recovers the mean full energy at fixed external field \(B\). Likewise, the susceptibility with respect to \(B\), at fixed \(\beta\), is
The factor \(\beta\) is absent from \(\partial_hp_N=m_N\) because \(h\) already includes the inverse-temperature factor. Both conventions describe the same Gibbs measure when \(h=\beta B\).
A function is real analytic if, near every point of its domain, it agrees with a convergent power series. In one variable, near \(t_0\), this means
for all \(t\) in some open interval containing \(t_0\). For a function of \((\beta,h)\), the corresponding local expansion near \((\beta_0,h_0)\) is
with a convergent power series on a neighborhood of that point. Exponentials, finite sums, and compositions of real analytic functions are real analytic wherever the compositions are defined. The neighborhood of convergence may depend on the point and on \(N\). The power series can be differentiated term by term: the differentiated series converge uniformly on every compact subset strictly inside the domain of convergence to the corresponding derivatives of \(q\); in one variable, they have the same radius of convergence as the original series.
Let \(X\) be a real random variable with finite variance. On the event \(\{|X-\mathbb E[X]|>\varepsilon\}\), its squared deviation is at least \(\varepsilon^2\). Thus
Taking expectations and dividing by \(\varepsilon^2>0\) gives
For an average of \(N\) independent spins, the variance is \(v_h/N\), which gives the explicit WLLN bound.
For \(M>N\), let \(\pi_N^M:\Omega_M\to\Omega_N\) retain the first \(N\) coordinates. A family of finite-dimensional laws can be the marginals of one infinite law only if
This is called consistency. For the product laws \(\mu_M=\rho_{0,h}^{\otimes M}\), it follows from the fact that the one-spin probabilities sum to one. To see precisely what “summing out” means, fix the first \(M-1\) spin values and sum over the two possible values of the last spin:
The first \(M-1\) factors do not depend on \(s_M\), so they can be taken outside the sum. The bracket is \(1\), because \(\rho_{0,h}\) is a probability law. Thus eliminating the last coordinate replaces its one-spin factor by \(1\): we add the probabilities of its two possible values, rather than merely count those values.
Repeat this operation for \(s_{M-1},\ldots,s_{N+1}\). Each eliminated spin contributes another factor \(1\), and only the factors for the first \(N\) spins remain. For every \(a=(a_1,\ldots,a_N)\in\Omega_N\), this gives
For a general set \(A\subseteq\Omega_N\), sum this identity over all \(a\in A\); the corresponding events are disjoint. This proves consistency. The infinite product measure \(\mu=\rho_{0,h}^{\otimes\mathbb N}\) has exactly these finite marginals.