# LOG#044. Hydrodynamics and SR(I).

Relativistic hydrodynamics is a branch of Relativity Theory that faces with fluids and/or molecules (“gases”) moving at relativistic speeds. Today, this area of Special Relativity has been covered with many applications. However, it has not been so since, not so long ago, the questions was:

Where could one encounter fluids or “gases” that would propagate with velocities close to the the speed of light?

It was thought that it seemed a question to be very far away from any realistic or practical use. At present time, relativistic hydrodynamics IS an importan part of Cosmology and the theory of processes going on in the sorrounding and ambient space of neutron stars (likely, of the quark stars as well), compact massive objects and black holes. When the relativistic fluid flows under strong gravitational fields existing in those extreme conditions at relativistic speeds, it drives to a big heating and X-ray emission, for instance. And then, a relativistic treatment of matter is inevitable there.

Caution note: I will use units with c=1 in this post, in general, without loss of generality.

Let me review a bit the non-relativistic hydrodynamics of ideal fluids and gases. Their dynamics is governed by the continuity equation (mass conservation) and the Euler equation:

$\boxed{\mbox{Continuity equation:\;\;}\dfrac{\partial \rho}{\partial t}+\nabla \cdot(\rho \mathbf{v})=0}$

$\boxed{\mbox{Euler equation:\;\;}\rho \dfrac{d\mathbf{v}}{dt}+\nabla p=\mathbf{f}}$

where the mass density and pressure of the fluid are respectively $\rho$ and $p$. To complete the fluid equations, these equations need to be supplemented by an equation of state:

$\boxed{\mbox{Equation of state:\;\;}p=p(\rho)}$

The continuity equation expresses the fact that mass in an invariant in classical fluid theory. The Euler equation says how the changes of pressure and forces affect to the velocity of the fluid, and finally the equation of state encodes the type of fluid we have at macroscopic level from the microscopic degrees of freedom that fluid theory itself can not see.

By the other hand, it is worth mentioning that we can not write the continuity equation into a covariant form $\partial_\mu j^\mu=0$ in such “a naive” way, with $j^\mu=\rho (x)v^\mu$. Why? It is pretty easy: it is a characteristic property of special relativity that the mass density $\rho (x)$ does not satisty such an equation, but we can derive a modified continuity equation that holds in SR. To build the right equations, we can proceed using an analogy with the electromagnetic field. Suppose we write an “stress-energy-momentum” tensor for an ideal fluid in the following way:

$T_{\mu\nu}=\begin{pmatrix}\rho & 0 & 0& 0\\ 0 & p & 0 & 0\\ 0 & 0 & p & 0\\ 0 & 0 & 0 & p \end{pmatrix}$

This tensor is written in the rest frame of a fluid. Note, then, that ideal fluids are characterized b the feature that their stress tensor $T_{ij}$ contains no shear stresses ( off-diagonal terms) and they are thus “proportional” to the Kronecker delta tensor $\delta_{\mu\nu}$.

The generalization of the above tensor (an ideal fluid) to an arbitrary reference frame, in which the fluid element moves with some 4-velocity components $u^\mu$ is given by the next natural generalization:

$T_{\mu\nu}=(\rho +p)u_\mu u_\nu-\eta_{\mu \nu}p$

where again, $\rho (x)$ represents the density, $p(x)$ is the local fluid pressure field as measured in the rest frame of the fluid element, and $\eta_{\mu\nu}$ is the Minkovski metric. Therefore, the equations of motion can be found, in the absence of external forces, from the conservation laws:

$\boxed{\mbox{Relativistic continuity equation:\;\;}\partial_\nu T^{\mu\nu}=0}$

Inserting the above tensor for the fluid, we get

$\partial_\nu \left[ \left( \rho +p\right)u^\mu u^\nu\right]-\partial^\mu p=0$

We can find this equation to the non-relativistic Euler equation. The trick is easy: we firstly multiply by $u^\mu$ and, after a short calculation, as we have that $u^\mu u_\mu=-1$, and $(\partial_\nu u^\mu) u_\mu=0$, it provides us with

$\partial_\mu(\rho u^\mu)+p\partial_\mu u^\mu=0$

This last equation shows, indeed, that mass current $\rho u^\mu$ is not conserved itself. Recall that from the spatial part of our relativist fluid equations:

$\partial_i \left[ \left( \rho +p\right)\mathbf{v} u^i\right]-\nabla p=0$

If we define the so-called “convective” or “comoving” derivative of an arbitrary tensor field T as the derivative:

$\dot{T}=\dfrac{DT}{d\lambda}=\partial_i Tu^i$

we can rewrite the spatial part of the relativistic fluid dynamics as follows:

$\boxed{\mbox{Relativistic Euler equation in 3d-space:\;}\dot{p}\mathbf{v}+(\rho+p)\dot{\mathbf{v}}+\nabla p=0}$

We can check that it effectively corresponds to the classical Euler equation moving to the comoving frame where $u^\mu=(1,\mathbf{0})^T$, excepting for a pressure term equals to $p/c^2$ if we reintroduce units with the speed of light, and then it is the generalized mass-energy density conservation law from relativistic hydrodynamics!

Remark: In the case of electromagnetic radiation, we have a pressure term equals to $p=\rho/3$ due to the tracelessneess of the electromagnetic stress-energy-momentum tensor.

Returning to our complete relativistic equation, we observe that the time component of that equation has NOT turned out to be the relativistic version of the continuity equation, as we warned before. The latter rather has to be postulated separately, using additional insights from elementary particle physics! In this way, instead of a mass density conservation, we do know, e.g., that the baryon density $n(x)$ does satisfy an equation of continuity (at least from current knowledge of fundamental physics):

$\partial_\mu (n u^\mu)=0$

and it merely says that baryon number is conserved under reasonable conditions ( of course, we do suspect at current time that baryon number conservation is not a good symmetry in the early Universe, but today it holds with good accuracy). Similarly, it can be said that for an electron gas, the baryon density has to be replaced by the so-called lepton density in the equation of continuity, where we could consider a gas with electrons, muons, tauons and their antiparticles. We could guess the neutrino density as well with suitable care. However, for phtoons and “mesons” there is NO continuity equation since tehy can be created and annihilated arbitrarily.

The relationship between n, p and $\rho$ can be obtained from the equation of state and basic thermodynamics from the definition of pressure:

$p=-\dfrac{\mbox{Energy per baryon}}{\mbox{volume per baryon}}=-\dfrac{(\rho/n)}{(1/n)}=n\dfrac{d\rho}{dn}-\rho$

or

$\int \dfrac{d\rho}{p(\rho)+\rho}=\int \dfrac{dn}{n}$

Thus, with these equations, we can know the density $n(\rho)$. The mass density $\rho$ and the baryon density $n$ differ by the density $n\epsilon$ of the inner energy ( we have defined $\epsilon$ as the specific inner energy or equivalently the inner energy per baryon):

$\rho=n(1+\epsilon)$

The inner energy is negative if energy is released at the formation of the state $\rho$, e.g. in the binding energy of a nucleus, and it is positive if energy has to be spent, e.g. if we make a compressional work onto the state.

Moreover, the specific entropy $s=S/B$ or entropy per baryon and the temperature are defined by postulating the thermodynamical equilibrium and, with that integrating factor $1/T$ coming from elementary thermodynamics, we can write

$ds=\dfrac{1}{T}\left(d\epsilon+pd\left(\dfrac{1}{n}\right)\right)$

since we have an specific volume equal to $v=1/n$. Entropy is constant along a stream line of the ideal fluid, and it follows from

$\dot{p}=\partial_\mu\left[(p+\rho)u^\mu\right]=\partial_\mu\left[(n+\epsilon n+p)u^\mu\right]=n\dot{\epsilon}+p\partial_\mu u^\mu+\dot{p}$

if we divide by n

$T\dot{s}=\dot{\epsilon}+p\left(\dfrac{1}{n}\right)$

In conclusion, we deduce that the time conservation of the relativistic conservation law of fluid hydrodynamics tell us that in the case of an ideal fluid no energy is converted into heat, and entropy itself remains constant! Isn’t it amazing? Entropy rocks!

Final remark (I): For non-ideal fluids, the ansatz for the energy-momentum-stress tensor hast to be modified into this one

$T_{\mu\nu}=(\rho+p)u_\mu u_\nu+(q_\mu u_\nu+q_\nu u_\mu)-\eta_{\mu\nu}p-\pi_{\mu\nu}$

Final remark (II): Relativistic hydrodynamics can be generalized to charged fluids and plasmas, and in that case, it is called relativistic magnetohydrodynamics. One simply adds the stress-energy-momentum tensor of the electromagnetic field to the tensor of the fluid in such a way it is conserved as well, i.e., with zero divergence. Thus, we would get an extra Lorentz-like force into the relativistic generalization of the Euler equation!

Final remark (III): The measurement of thermodynamical quantities like pressure, entropy or temperature, and its treatment with classical thermodynamics suppose that the thermodynamical equilibrium state is reached. Please, note that if it were not the case, we should handle the problem with tools of non-equilibrium thermodynamics or some other type of statistical mechanics that could describe the out-of-the equilibrium states ( there are some suggestions of such a generalization of thermodynamics, and/or statistical mechanics, from the works of I. Prigogine, C.Tsallis, C.Beck and many other physicists).

Hello, I am back! After some summer rest/study/introspection! And after an amazing July month with the Higgs discovery by ATLAS and CMS. After an amazing August month with the Curiosity rover, MSL(Mars Science Laboratory), arrival to Mars. After a hot summer in my home town…I have written lots of drafts these days…And I will be publishing all of them step to step.

We will discuss today one of interesting remark studied by Kaniadakis. He is known by his works on relatistivic physics, condensed matter physics, and specially by his work on some cool function related to non-extensive thermodynamics. Indeed, Kaniadakis himself has probed that his entropy is also related to the mathematics of special relativity. Ultimately, his remarks suggest:

1st. Dimensionless quantities are the true fundamental objects in any theory.

2nd. A relationship between information theory and relativity.

3rd. The important role of deformation parameters and deformed calculus in contemporary Physics, and more and more in the future maybe.

4nd. Entropy cound be more fundamental than thought before, in the sense that non-extensive generalizations of entropy play a more significant role in Physics.

5th. Non-extensive entropies are more fundamental than the conventional entropy.

The fundamental object we are going to find is stuff related to the following function:

$exp_\kappa (x)=\left( \sqrt{1+\kappa^2x^2}\right)^{1/\kappa}$

Let me first imagine two identical particles ( of equal mass) A and B, whose velocities, momenta and energies are, in certain frame S:

$v_A, p_A=p(v_A), E_A=E(v_A)$

$v_B, p_B=p(v_B), E_B=E(v_B)$

In the rest frame of particle B, S’, we have

$p'_B=0$

$p'_A=p_A-p_B$

If we define a dimensionless momentum paramenter

$q=\dfrac{p}{p^\star}$

$\dfrac{p'_A}{p^\star}=\dfrac{p_A}{p^\star}-\dfrac{p_B}{p^\star}$

we get after usual exponentiation

$\exp(q'_A)=\exp(q_A)\exp(-q_B)$

Galilean relativity says that the laws of Mechanics are unchanged after the changes from rest to an uniform motion reference frame. Equivalentaly, galilean relativity in our context means the invariance under a change $q'_A\leftrightarrow q_A$, and it implies the invariance under a change $q_B\rightarrow -q_B$. In turn, plugging these inte the last previous equation, we get the know relationship

$\exp (q)\exp (-q)=1$

Wonderful, isn’t it? It is for me! Now, we will move to Special Relativity. In the S’ frame where B is at rest, we have:

$v'_B=0, p'_B=0, E'_B=mc^2$

and from the known relativistic transformations for energy and momentum

$v'_A=\dfrac{v_A-v_B}{1-\dfrac{v_Av_B}{c^2}}$

$p'_A=\gamma (v_B)p_A-\dfrac{v_B\gamma (v_B)E_A}{c^2}$

$E'_A=\gamma (v_B)E_A-v_B\gamma (v_B)p_A$

where of course we define

$\gamma (v_B)=\dfrac{1}{\sqrt{1-\dfrac{v_B}{c^2}}}$

$p_B=m \gamma (v_B) v_B$

$E_B=m \gamma (v_B) c^2$

After this introduction, we can parallel what we did for galilean relativity. We can write the last previous equations in the equivalent form, after some easy algebra, as follows

$p'_A=p_A\dfrac{E_B}{mc^2}-E_A\dfrac{p_B}{mc^2}$

$E'_A=E_AE_B\dfrac{1}{mc^2}-\dfrac{p_Ap_B}{m}$

Now, we can introduce dimensionless variables instead of the triple $(v, p, E)$, defining instead the adimensional set $(u, q, \epsilon)$:

$\dfrac{v}{u}=\dfrac{p}{mq}=\sqrt{\dfrac{E}{m\epsilon}}=\vert \kappa \vert c=v_\star

Note that the so-called deformation parameter $\kappa$ is indeed related (equal) to the beta parameter in relativity. Again, from the special relativity requirement $\vert \kappa \vert c we obtain, as we expected, that $-1< \kappa <+1$. Classical physics, the galilean relativity we know from our everyday experience, is recovered in the limit $c\rightarrow \infty$, or equivalently, if $\kappa \rightarrow 0$. In the dimensionless variables, the transformation of energy and momentum we wrote above can be showed to be:

$q'_A=\kappa^2q_A\epsilon_B-\kappa^2q_B\epsilon_A$

$\epsilon'_A=\kappa^2\epsilon_A\epsilon_B-q_Aq_B$

In rest frame of some particle, we get of course the result $E(0)=mc^2$, or in the new variables $\epsilon (0)=\dfrac{1}{\kappa^2}$. The energy-momentum dispersion relationship from special relativity $p^2c^2-E^2=-m^2c^2$ becomes:

$q^2-\kappa^2\epsilon^2=-\dfrac{1}{\kappa^2}$

or

$\kappa^4\epsilon^2-\kappa^2q^2=1$

Moreover, we can rewrite the equation

$q'_A=\kappa^2q_A\epsilon_B-\kappa^2q_B\epsilon_A$

in terms of the dimensionless energy-momentum variable

$\epsilon_\kappa (q)=\dfrac{\sqrt{1+\kappa^2q^2}}{\kappa^2}$

amd we get the analogue of the galilean addition rule for dimensionless velocities

$q'_A =q_A\sqrt{1+\kappa^2q_B^2}-q_B\sqrt{1+\kappa^2q_A^2}$

Note that the classical limit is recovered again sending $\kappa\rightarrow 0$. Now, we have to define some kind of deformed exponential function. Let us define:

$\exp_\kappa (q) =\left(\sqrt{1+\kappa^2q^2}+\kappa q\right)^{1/\kappa}$

Applying this function to the above last equation, we observe that

$\exp_\kappa (q'_A)=\exp_\kappa (q_A) \exp_\kappa (-q_B)$

Again, relativity means that observers in uniform motion with respect to each other should observe the same physical laws, and so, we should obtain invariant equations under the exchanges $q'_A\leftrightarrow q_A$ and $q_B\rightarrow -q_B$. Pluggint these conditions into the last equation, it implies that the following condition holds (and it can easily be checked from the definition of the deformed exponential).

$\exp_\kappa (q)\exp_\kappa (-q)=1$

One interesting question is what is the inverse of this deformed exponential ( the name q-exponential or $\kappa$-exponential is often found in the literature). It has to be some kind of deformed logarithm. And it is! The deformed logarithm, inverse to the deformed exponential, is the following function:

$\ln_\kappa (q)=\dfrac{q^{\kappa}-q^{-\kappa}}{2\kappa}$

Indeed, this function is related to ( in units with the Boltzmann’s constant set to the unit $k_B=1$) the so-called Kaniadakis entropy!

$S_{K}=-\dfrac{q^{\kappa}-q^{-\kappa}}{2\kappa}$

Furthermore, the equation $\exp_\kappa (q)\exp_\kappa (-q)=1$ also implies that

$\ln_\kappa \left(\dfrac{1}{q}\right)=-\ln_\kappa (q)$

The gamma parameter of special relativity is also recasted as

$\gamma =\dfrac{1}{\sqrt{1-\kappa^2}}$

More generally, in fact, the deformed exponentials and logarithms develop a complete calculus based on:

$\exp_\kappa (q_A)\exp_\kappa (q_B)=\exp (q_A\oplus q_B)$

and the differential operators

$\dfrac{d}{d_\kappa q}=\sqrt{1+\kappa^2q^2}\dfrac{d}{dq}$

so that, e.g.,

$\dfrac{d}{d_\kappa q}\exp_\kappa (q)=\exp_\kappa (q)$

This Kanadiakis formalism is useful, for instance, in generalizations of Statistical Mechanics. It is becoming a powertool in High Energy Physics. At low energy, classical statistical mechanics gets a Steffan-Boltmann exponential factor distribution function:

$f\propto \exp(-\beta E)=\exp (-\kappa E)$

At high energies, in the relativistic domain, Kaniadakis approach provide that the distribution function departures from the classical value to a power law:

$f\propto E^{-1/\kappa}$

There are other approaches and entropies that could be interesting for additional deformations of special relativity. It is useful also in the foundations of Physics, in the Information Theory approach that sorrounds the subject in current times. And of course, it is full of incredibly beautiful mathematics!

We can start from deformed exponentials and logarithms in order to get the special theory of relativity (reversing the order in which I have introduced this topic here). Aren’t you surprised?

# LOG#003. Entropy.

“I propose to name the quantity S the entropy of the system, after the Greek word [τροπη trope], the transformation. I have deliberately chosen the word entropy to be as similar as possible to the word energy: the two quantities to be named by these words are so closely related in physical significance that a certain similarity in their names appears to be appropriate.”  Clausius (1865).

Entropy is one of the strangest and wonderful concepts in Physics. It is essential for the whole Thermodynamics and it is essential also to understand thermal machines. It is essential for Statistical Mechanics and the atomic structure of molecules and fundamental particles. From the Microcosmos to the Macrocosmos, entropy is everywhere: from  the kinetic theory of gases, information theory as we learned from the previous post, and it is also relevant in the realm of General Relativity, where equations of state for relativistic and non-relativistic particles arise too. And even more, entropy arises in the Black Hole Thermodynamics in a most mysterious form that nobody understands yet.

By the other hand, in the Quantum Mechanics, entropy arises in the (Von Neumann’s) approach to density matrix, the quantum incarnation of the classical version of entropy, ultimately related to the notion of  quantum entanglement. I have no knowledge of any other concept in Physics that can appear in such diffent branches of Physics. The true power of the concept of entropy is its generality.

There are generally three foundations for entropy, three roads to the entropy meaning that physicists have:

– Thermodynamical Entropy. In Thermodynamics, entropy arises after integrating out the heat with an integrating factor that is nothing but the inverse of the temperature. That is:

$\boxed{dS=\oint_\gamma\dfrac{\delta Q}{T}\rightarrow \Delta S= \dfrac{\Delta Q}{T}}$

The studies of thermal machines that existed as logical consequence of the Industrial Revolution during the XIX century created the first definition of entropy. Indeed, following Clausius, the entropy change $\Delta S$ of a thermodynamic system absorbing a quantity of heat $\Delta Q$  at absolute temperature T is simply the ratio between the two, as the above formula shows!  Armed with this definition and concept, Clausius was able to recast Carnot’s statement that steam engines can not exceed a specific theoretical optimum efficiency into a much grander principle we do know as the “2nd law of Thermodynamics” (sometimes called The Maximum Entropy, MAX-ENT, principle by other authors)

$\boxed{\mbox{The entropy of the Universe tends to a maximum}}$

The problem with this definition and this principle is that  it leaves unanswered the most important questionwhat really is the meaning of entropy? Indeed, the answer to this question had to await the revival of  atomic theories of the matter at the end of the 19th century.

– Statistical Entropy.  Ludwig Boltzmann was the scientist who provided a fundamental theoretical basis to the concept of entropy. His key observation was that absolute temperature is nothing more than the average energy per molecular degree of freedom. This fact strongly implies that Clausius ratio between absorbed energy and absolute temperature is nothing more than the number of molecular degrees of freedom. That is, Boltzmann greatest idea was indeed very simply put into words:

$\boxed{S=\mbox{Number of microscopical degrees of freedom}= N_{dof}}$

We can see a difference with respect to the thermodynamical picture of entropy: Boltzmann was able to show that the number of degrees of freedom of a physical system can be easily linked to the number of microstates $\Omega$ of that system. And it comes with a relatively simple expression from the mathematical viewpoint (using the 7th elementary arithmetical operation, beyond the more known addition, substraction, multiplication, division, powers, roots,…)

$\boxed{S \propto \log \Omega}$

Really the base of the logarithm is absolutely conventional. Generally, it is used the natural base (or the binary base, see below).

Why does it work? Why is the number of degrees of freedom related to the logarithm of the total number of available microscopical states? Imagine a system with 2 simple degrees of freedom, a coin. Clone/copy it up to N of those systems. Then, we have got a system of N coins  showing head or tail. Each coin contributes one degree of freedom that can take two distinct values. So in total we have N (binary, i.e., head or tail) degrees of freedom. Simple counting tells us that each coin (each degree of freedom) contributes a factor of two to the total number of distinct states the system can be in. In other words, $\Omega = 2^N$.  Taking the base-2 logarithm  of both sides of this equation yields the logarithm of the total number of states to equal the number of degrees of freedom: $\log_2 \Omega = N$.

This argument can be made completely general. The key argument is that the total number of states  $\Omega$ follows from multiplying together the number of states for each degree of freedom. By taking the logarithm of  $\Omega$, this product gets transformed into an addition of degrees of freedom. The result is an additive entropy definition: adding up the entropies of two independent subsystems provides us the entropy of the total system.

– Information Entropy.
Time machine towards the past future. 20th century. In 1948, Claude Shannon, an electrical engineer at Bell Telephone Laboratories, managed to mathematically quantify the concept of “information”. The key result he derived is that to describe the precise state of a system that can be in states labelled by numbers $1,2,...,n$ with probabilities $p_1, p_2,...,p_n$.

It requires a well-defined minimum number of bits. In fact, the best one can do is to assign $\log_2 (1/p_i)$ bits to the one event with state $i$. Result:  statistically speaking the minimum number of bits one needs to be capable of specifying the system regardless its precise state will be

$\displaystyle{\mbox{Minimum number of bits} = \sum_{i=i}^{n}p_i\log_2 p_i = p_1\log_2 p_1+p_2\log_2 p_2+...+p_n\log_2 p_n}$

When applied to a system that can be in $\Omega$ states, each with equal  probability $p= 1/\Omega$, we get that

$\mbox{Minimum number of bits} = \log_2 \Omega$

We got it. A full century after the thermodynamic and statistical research we were lead to the simple conclusion that the Boltzmann expression $S = \log \Omega$ is nothing more than an alternative way to express:

$S = \mbox{number of bits required to define some (sub)system}$

Entropy is therefore a simple bit (or trit, cuatrit, pentit,…,p-it) counting of your system. The number of bits required to completely determine the actual microscopic configuration between the total number of microstates allowed. In these terms the second law of thermodynamics tells us that closed systems tend to be characterized by a growing bit count. Does it work? Yes, it does. Very well as far as we know…Even in quantum information theory you have an analogue with the density matrix. Even it works in GR and even it strangely works with Black Hole Thermodynamics, excepting the fact that entropy is the area of the horizon, temperature is the surface gravity in the horizon, and the fact that mysteriously, BH entropy is proportional not to the volume as one could expect from conventional thermodynamics (where entropy scales as the volume of the container) , but to the area of the horizon. Incredible, isn’t it? That scaling of the Black Hole entropy with the area was the origin of the holographic principle. But it is far away where my current post wants to go today.

Indeed, there is  a subtle difference between the statistical and the informational entropy. A sign minus in the definition. (Thermodynamical) Entropy can be understood as “missing” (information) entropy:

$\boxed{Entropy = - Information}$

or mathematically

$S= - I$, do you prefer maybe $I+S=0$?

That is, entropy is the same thing that information, excepting a minus sign! So, if you add the same thing to its opposite you get zero.

The question that naturally we face in this entry is the following one: what is the most general mathematical formula/equation for “microscopic” entropy? Well, as many others great problems in Physics, it depends on the axiomatics and your assumptions! Let’s follow Boltzmann during the XIX century. He cleverly suggested a deep connection between thermodynamical entropy and the microscopical degrees of freedom of the considered system. He suggested that there were a connection between the entropy S of a thermodynamical system and the probability $\Omega$ of a given thermodynamical state. How can the functional relationship between S and $\Omega$ be found? Suppose we have $S=f(\Omega)$. In addition, suppose that we have a system that can be divided into two pieces, with their respective entropies and probabilities $S_1,S_2,\Omega_1,\Omega_2$. If we assume that the entropy is additive, meaning that

$S_\Omega=S_1(\Omega_1)+S_2(\Omega_2)$

with the additional hypothesis that the sybsystems are independent, i.e., $\Omega=\Omega_1\Omega_2$, then we can fix the functional form of the entropy in a very easy way: $S(\Omega)=f(\Omega_1\Omega_2)=f(\Omega_1)+f(\Omega_2)$. Do you recognize this functional equation from your High-School? Yes! Logarithms are involved with it. If you are not convinced, with simple calculus, following the Fermi lectures on Thermodynamics you can do the work. Let be $x=\Omega_1, y=\Omega_2$

$f(xy)=f(x)+f(y)$

Write now $y=1+\epsilon$, then $f(x+\epsilon x)=f(x)+f(1+\epsilon)$, where $\epsilon$ is a tiny infinitesimal quantity of first order. Thus, Taylor expanding to both sides, neglecting terms higher to first order infinitesimals, we get

$f(x)+\epsilon f'(x)=f(x)+f(1)+\epsilon f'(1)$

For $\epsilon=0$ we obtain $f(1)=0$, and therefore $xf'(x)=f'(1)=k=constant$, where k is a constant, and nowadays it is  called Boltzmann’s constant. We integrate the differential equation:

$f'(x)=k/x$ in order to obtain the celebrated Boltzmann’s equation for entropy: $S=k\log \Omega$. To be precise, $\Omega$ is not the probability, is the number of microstates compatible with the given thermodynamical state. To obtain the so-called Shannon-Gibbs-Boltzmann entropy, we must divide $\Omega$ between the number of possible dynamical states that agree with the microstate.The Shannon entropy functional form is then generally written as follows:

$\displaystyle{\boxed{S=-k \sum_i p_i\log p_i}}$

It approaches a maximum value when $p_i=1/\Omega$, i.e., when the probability$\Omega$ is a uniform distribution. There is a subtle issue related to the additive constant obtained from the above argument that is important in classical and quantum thermodynamics. But we will discuss that in the future. Now, we could be happy with this functional entropy but indeed, the real issue is that we derived it from some a priori axioms that could look natural, but they are not the most general set of axioms. And, then,  our fascinating trip continues here today! There previous considerations have been using, more or less, formal according to the so-called  “Khinchin axioms” of information theory. That is, The Khinchin axioms are enough to derive the Shannon-Gibbs-Boltzmann entropy we wrote before. However, as it happened with the axioms of euclidean geometry, we can modify our axioms in order to obtain more general “geometries”, here more general “statistical mechanics”. We are going now to explore some of the most known generalizations to Shannon entropy.In the succesive, for simplicity, we set the Boltzmann’s constant to one (i.e. we work with a k=1 system of units ). Is the above definition of entropy/information the only one that is interesting from the physical viewpoint? No, indeed, there has been an increasing activity in “generalized entropies” in the past years. Note, however, that we should recover the basic and simpler entropy (that of Shannon-Gibbs-Boltzmann) in some limit. I will review here some of the most studied entropic functionals that have been studied during the last decades.

The Rényi entropy.

It is a set of uniparametric entropies, now becoming more and more popular in works on entanglement and thermodynamics, with the following functional form:

$\displaystyle{ \boxed{S_q^R=\dfrac{1}{1-q}\ln \sum_i p_{i}^{q}}}$

where the sums extends itself to any microstate with non zero probability $p_i$. It is quite easy to see that in the limit $q\rightarrow 1$ the Rényi entropy transforms into the Shannon-Gibbs-Boltzmann entropy (it can be checked with a perturbative expansion around $q=1+\epsilon$ or using the L’Hôspital’s rule.

The Tsallis entropy.

Tsallis entropies, also called q-entropies by some researchers, are the uniparametric family of entropies defined by:

$\displaystyle{ \boxed{S_{q}^{T}=\dfrac{1}{1-q}\left( \sum_{i} p_{i}^{q}-1\right)}}$.

Tsallis entropy is related to Rényi’s entropies throug a nice equation:

$\boxed{S_q^ T=\dfrac{1}{q-1}(1-e^{(q-1)S_q^R})}$

and again, taking the limit q=1, Tsallis entropies provide the Shannon-Gibbs-Boltzmann’s entropies. Why consider such a Statistical Mechanics based on Tsallis entropy and not Renyi’s?Without entrying into mathematical details, the properties of Tsallis entropy makes itself more suitable to a generalized Statistical Mechanics for complex systems(in particular, it is due to the concavity of Tsallis entropy), as the seminal work of C.Tsallis showed. Indeed, Tsallis entropies were found unnoticed by Tsallis in other unexpected place. In a paper, Havrda-Charvat introduced the so-called “structural $\alpha$ entropy” related to some cybernetical problems in computing.

Interestingly, Tsallis entropies are non-additive, meaning that they satisfy a “pseudo-additivity” property:

$\boxed{S_{q}^{\Omega}=S_q^{\Omega_1}+S_q^{\Omega_2}-(q-1)S_q^{\Omega_1}S_q^{\Omega_2}}$

This means that if we built a Statistical Mechanics based on the Tsallis entropy, it is non-additive itself. Independent subsystems are generally non-additive. However, they are usually called “non-extensive” entropies. Why? The definition of extensivity is  different, namely the entropy of a given system is extensive if, in the so called thermodynamicla limit $N\rightarrow \infty$, then $S\propto N$ , where N is the number of elements of the given thermodynamical system. Therefore, the additivity only depends on the functional relation between the entropy and the probabilities, but extensivity depends not only on that, but also on the nature of the correlations between the elements of the system. The entropic additivity test is quite trivial, but checking its extensivity for a specific system can be complex and very complicated. Indeed, Tsallis entropies can be additive for certain systems, and for some correlated systems, they can become extensive, like usual Thermodynamics/Statistical Mechanics. However, in the broader sense, they are generally non-additive and non-extensive. And it is the latter feature, its thermodynamical behaviour in the thermodynamical limit, from where the name “non-extensive” Thermodynamics arises.

Landsberg-Vedral entropy.

They are also called “normalized Tsallis entropies”. Their functional form are the uniparametric family of entropies:

$\displaystyle{ \boxed{S_q^{LV} =\dfrac{1}{1-q} \left( 1-\dfrac{1}{\sum_i p_{i}^{q}}\right)}}$

They are related to Tsallis entropy through the equation:

$\displaystyle{ S_q^{LV}= \dfrac{S_q^T}{\sum_i p_i ^q}}$

It explains their alternate name as “normalized” Tsallis entropies. They satisfy a modified “pseudoadditivity” property:

$S_q^\Omega=S_q^{\Omega_1}+S_q^{\Omega_2}+(q-1)S_q^{\Omega_1}S_q^{\Omega_2}$

That is, in the case of normalized Tsallis entropies the rôle of (q-1) and -(q-1) is exchanged, i.e., -(q-1) becomes (q-1) in the transition from Tsallis to Landsberg-Vegral entropy.

Abe entropy.

This kind of uniparametric entropy is very symmetric. It is also related to some issues in quantum groups and fractal (non-integer) analysis. They are defined by the q-1/q entropic functional:

$\displaystyle{ \boxed{S_q^{Abe}=-\sum_i \dfrac{p_i^q-p_i^{q^{-1}}}{q-q^{-1}}}}$

Abe entropy can be obtained from Tsallis entropy as follows:

$\boxed{S_q^{ LV}=\dfrac{(q-1)S_q^T-(q^{-1}-1)S_{q^{-1}}^{T}}{q-q^{-1}}}$

Abe entropy is also concave from the mathematical viewpoint, like the Tsallis entropy. It has some king of “duality” or mirror symmetry due to the invariance swapping q and 1/q.

Other uniparametric entropic family well-know is the Kaniadakis entropy or $latex \kappa$-entropy. Related to relativistic kinematics, it has the functional form:

$\displaystyle{ \boxed{S_\kappa^{K}=-\sum_i \dfrac{p_i^{1+\kappa }-p_i^{1-\kappa}}{2\kappa}}}$

In the limit $\kappa \rightarrow 0$ Kaniadakis entropy becomes Shannon entropy. Also, writing $q=1+\kappa$, and $\dfrac{1}{q}=1-\kappa$, Kaniadakis entropy becomes Abe entropy. Kaniadakis entropy, in addition to be concave, have further subtle properties, like being something called Lesche stable. See references below for details!

Sharma-Mittal entropies.

Finally, we end our tour along entropy functionals with a biparametric family of entropies called Sharma-Mittal entropies. They have the following definition:

$\displaystyle{ \boxed{S_{\kappa,r}^{SM}=-\sum_i p_i^{r}\left( \dfrac{p_i^{1+\kappa}-p_i^{1-\kappa}}{2\kappa}\right)}}$

It can be shown that these entropy species contain many entropies as special subtypes. For instance, Tsallis entropy is recovered if $r=\kappa$ and $q=1-2\kappa$. Kaniadakis entropy is got if we set r=0. Abe entropy is the subcase with $\kappa=\frac{1}{2}(q-q^{-1})$ and $r=\frac{1}{2}(q+q^{-1})-1$. Isn’t it wonderful? There is an alternative expression of Sharma-Mittal entropy, taking the following expression:

$\displaystyle{ \boxed{S_{r,q}^{SM}=\dfrac{1}{1-r}\left[\sum_i \left(p_i^q\right)^{(\frac{1-r}{1-q})}-1\right]}}$

In this functional form, SM entropy recovers Rényi entropy for $r\rightarrow 1$, SM entropy becomes Tsallis entropy if $r\rightarrow q$. Finally, when both parameters approach 1, i.e., $r,q\rightarrow 1$, we recover the classical Shannon-Gibbs-Boltzmann. It is left as a nice exercise for the reader to relate the above 2 SM entropy functional forms and to derive Kaniadakis entropy, Abe entropy and Landsberg-Vedral entropy for some particular values of $r,q$ from the second definition of SM entropy.

However, entropy as a concept is yet very mysterious. Indeed, it is not clear yet if we have exhausted every functional form for entropy!

Non-extensive Statistical Mechanics and its applications are becoming more and more important and kwown between the theoretical physicists. It has a growing number of uses in High-Energy Physics, condensed matter, Quantum Information and Physics. The Nobel Prize Murray Gell-Mann has dedicated their last years of research in the world of Non-Extensive entropy. At least, from his book The Quark and the Jaguar, Murray Gell-Mann has progressively moved into this fascinating topic. In parallel, it has also produced some other interesting approaches to Statistical Mechanics, such as the so-called “superstatistics”. Superstatistics is some kind of superposition of statistics that was invented by the physicist Christian Beck.

The last research on the foundations of entropy functionals is related to something called “group entropies” and the transformation group of superstatistics and the rôle of group transformations on non-extensive entropies. It provides feedback between different branches of knowledge: group theory, number theory, Statistical Mechanics, and Quantum Satisties…And a connection with the classical Riemann zeta function even arises!

WHERE DO I LEARN ABOUT THIS STUFF and MORE if I am interested in it? You can study these topics in the following references:

The main entry is based in the following article by Christian Beck:

1) Generalized information and entropy measures in physics by Christian Beck. http://arXiv.org/abs/0902.1235v2

If you get interested in Murray Gell-Mann works about superstatistics and its group of transformations, here is the place to begin with:

2) Generalized entropies and the transformation group of superstatistics. Rudolf Haner, Stefan Thurner, Murray Gell-Mann

http://arxiv.org/abs/1103.0580

If you want to see a really nice paper on group entropies and zeta functions, you can read this really nice paper by P.Tempesta:

3)Group entropies, correlation laws and zeta functions. http://arxiv.org/abs/1105.1935

C.Tsallis himself has a nice bibliography related to non-extensive entropies in his web page:

The “Khinchin axioms” of information/entropy functionals can be found, for instance, here:

5) Mathematical Foundations of Information Theory, A. Y. Khinchin. Dover. Pub.

Two questions to be answered by the current and future scientists:

A) What is the most general entropy (functional entropy) that can be build from microscopic degrees of freedom? Are they classical/quantum or is that distinction irrelevant for the ultimate substrate of reality?

B) Is every fundamental interaction related to some kind of entropy? How and why?

C) If entropy is “information loss” or “information” ( only a minus sign makes the difference), and Quantum Mechanics says that Quantum Mechanics is about information (the current and modern interpretation of QM is based on it), is there some hidden relationship between mass-energy and information and entropy? Could it be used to build Relativity and QM from a common framework? Therefore, are then QM and (General) Relativity emergent and likely the two sides of a most fundamental theory based on information only?

# LOG#002. Information and noise.

I  enjoyed as a teenager that old game in which you are told a message in your ear, and you transmit it to other human, this one to another and so on. Today, you can see it at big scale on Twitter. Hey! The message is generally very different to the original one! This simple example explains the other side of communication or information transmission: “noise”.  Although efficiency is also used. The storage or transmission of information is generally not completely efficient. You can loose information. Roughly speaking, every amount of information has some quantity of noise that depends upon how you transmit the information(you can include a noiseless transmission as a subtype of information process in which,  there is no lost information). Indeed, this is also why we age. Our DNA, which is continuously replicating itself thanks to the metabolism (possible ultimately thanksto the solar light), gets progressively corrupted by free radicals and  different “chemicals” that makes our cellular replication more and more inefficient. Don’t you remember it to something you do know from High-School? Yes! I am thinking about Thermodynamics. Indeed, the reason because Thermodynamics was a main topic during the 19th century till now, is simple: quantity of energy is constant but its quality is not. Then, we must be careful to build machines/engines that be energy-efficient for the available energy sources.

Before going into further details, you are likely wondering about what information is! It is a set of symbols, signs or objects with some well defined order. That is what information is. For instance, the word ORDER is giving you  information. A random permutation of those letters, like ORRDE or OERRD is generally meaningless. I said information was “something” but I didn’t go any further! Well, here is where Mathematics and Physics appear. Don’t run far away!  The beauty of Physics and Maths, or as I like to call them, Physmatics, is that concepts, intuitions and definitions, rigorously made, are well enough to satisfy your general requirements. Something IS a general object, or a set of objects with certain order. It can be certain DNA sequence coding how to produce certain substance (e.g.: a protein) our body needs. It can a simple or complex message hidden in a highly advanced cryptographic code. It is whatever you are recording on your DVD ( a new OS, a movie, your favourite music,…) or any other storage device. It can also be what your brain is learning how to do. That is  “something”, or really whatever. You can say it is something obscure and weird definition. Really it is! It can also be what electromagnetic waves transmit. Is it magic? Maybe! It has always seems magic to me how you can browse the internet thanks to your Wi-Fi network! Of course, it is not magic. It is Science. Digital or analogic information can be seen as large ordered strings of  1’s and 0’s, making “bits” of information. We will not discuss about bits in this log. Future logs will…

Now, we have to introduce the concepts through some general ideas we have mention and we know from High-School. Firstly, Thermodynamics. As everybody knows, and you have experiences about it, energy can not completely turned into useful “work”. There is a quality in energy. Heat is the most degradated form of energy. When you turn on your car and you burn fuel, you know that some of the energy is transformed into mechanical energy and a lof of energy is dissipated into heat to the atmosphere. I will not talk about the details about the different cycles engines can realize, but you can learn more about them in the references below. Simbollically, we can state that

$\begin{pmatrix} AVAILABLE \\ENERGY\end{pmatrix}=\begin{pmatrix}TOTAL \;\;ENERGY \\SUPPLIED\end{pmatrix} - \begin{pmatrix}UNAVAILABLE \\ENERGY\end{pmatrix}$

The great thing is that an analogue relation in information theory  does exist! The relation is:

$\boxed{\mbox{INFORMATION} = \mbox{SIGNAL} - \mbox{NOISE}}$

Therefore, there is some subtle analogy and likely some deeper idea with all this stuff. How do physicists play to this game? It is easy. They invent a “thermodynamic potential”! A thermodynamic potential is a gadget (mathematically a function) that relates a set of different thermodynamic variables. For all practical purposes, we will focus here with the so-called Gibbs “free-energy”. It allows to measure how useful a “chemical reaction” or “process” is. Moreover, it also gives a criterion of spontaneity for processes with constant pressure and temperature. But it is not important for the present discussion. Let’s define Gibbs free energy G as follows:

$G= H - TS$

where H is called enthalpy, T is the temperature and S is the entropy. You can identify these terms with the previous concepts. Can you see the similarity with the written letters in terms of energy and communication concepts? Information is something like “free energy” (do you like freedom?Sure! You will love free energy!). Thus, noise is related to entropy and temperature, to randomness, i.e., something that does not store “useful information”.

Internet is also a source of information and noise. There are lots of good readings but there are also spam. Spam is not really useful for you, isn’t it? Recalling our thermodynamic analogy, since the first law of thermodynamics says that the “quantity of energy” is constant and the second law says something like “the quality of energy, in general, decreases“, we have to be aware of information/energy processing. You find that there are signals and noise out there. This is also important, for instance, in High Energy Physics or particle Physics. You have to distinguish in a collision process what events are a “signal” from a generally big “background”.

We will learn more about information(or entropy) and noise in my next log entries. Hopefully, my blog and microblog will become signals and not noise in the whole web.

Where could you get more information? 😀 You have some good ideas and suggestions in the following references:

1) I found many years ago the analogy between Thermodynamics-Information in this cool book (easy to read for even for non-experts)

Applied Chaos Theory: A paradigm for complexity. Ali Bulent Cambel (Author)Publisher: Academic Press; 1st edition (November 19, 1992)

Unfortunately, in those times, as an undergraduate student, my teachers were not very interested in this subject. What a pity!

2) There are some good books on Thermodynamics, I love (and fortunately own) these jewels:

Concepts in Thermal Physics, by Stephen Blundell, OUP. 2009.

A really self-contained book on Thermodynamics, Statistical Physics and topics not included in standard books. I really like it very much. It includes some issues related to the global warming and interesting Mathematics. I enjoy how it introduces polylogarithms in order to handle closed formulae for the Quantum Statistics.

Thermodynamcis and Statistical Mechanics. (Dover Books on Physics & Chemistry). Peter T. Landsberg

A really old-fashioned and weird book. But it has some insights to make you think about the foundations of Thermodynamics.

Thermodynamcis, Dover Pub. Enrico Fermi

This really tiny book is delicious. I learned a lot of fun stuff from it. Basic, concise and completely original, as Fermi himself. Are you afraid of him? Me too! E. Fermi was a really exceptional physicist and lecturer. Don’t loose the opportunity to read his lectures on Thermodynamcis.

Mere Thermodynamics. Don S. Lemons. Johns Hopkins University Press.

Other  great little book if you really need a crash course on Thermodynamics.

Introduction to Modern Statistical Physics: A Set of Lectures. Zaitsev, R.O. URSS publishings.

I have read and learned some extra stuff from URSS ed. books like this one. Russian books on Science are generally great and uncommon. And I enjoy some very great poorly known books written by generally unknow russian scientists. Of course, you have ever known about Landau and Lipshitz books but there are many other russian authors who deserve your attention.

3) Information Theory books. Classical information theory books for your curious minds are

An Introduction to Information Theory: Symbols, Signals and Noise. Dover Pub. 2nd Revised ed. 1980.   John. R. Pierce.

A really nice and basic book about classical Information Theory.

An introduction to Information Theory. Dover Books on Mathematics. F.M.Reza. Basic book for beginners.

The Mathematical Theory of Communication. Claude E. Shannon and W.Weaver.Univ. of Illinois Press.

A classical book by one of the fathers of information and communication theory.

Mathematical Foundations of Information Theory. Dover Books on Mathematics. A.Y.Khinchin.

A “must read” if you are interested in the mathematical foundations of IT.