View source for Negentropy

{{Short description|Measure of distance to normality}}
{{Distinguish|text = [[Entropy and life#Negative entropy|Negative entropy]]{{Clarify|reason=The 'Negative entropy' article seems to deal with exactly the same material (Schrödinger's 1944 ''What is Life?'')|post-text=Should this be "[[template:for|For]] biological contexts..."?|date=December 2024}}}}
{{Redirect|Syntropy||Syntropy (software)}}
In [[information theory]] and [[statistics]], '''negentropy''' is used as a measure of distance to normality.  The concept and phrase "'''negative entropy'''" was introduced by [[Erwin Schrödinger]] in his 1944 popular-science book ''[[What is Life? (Schrödinger)|What is Life?]]''<ref>Schrödinger, Erwin, ''What is Life – the Physical Aspect of the Living Cell'', Cambridge University Press, 1944</ref> Later, [[French people|French]] [[physicist]] [[Léon Brillouin]] shortened the phrase to ''néguentropie'' (negentropy).<ref>Brillouin, Leon: (1953) "Negentropy Principle of Information", ''J. of Applied Physics'', v. '''24(9)''', pp. 1152–1163</ref><ref>Léon Brillouin, ''La science et la théorie de l'information'', Masson, 1959</ref> In 1974, [[Albert Szent-Györgyi]] proposed replacing the term ''negentropy'' with '''''syntropy'''''. That term may have originated in the 1940s with the Italian mathematician [[Luigi Fantappiè]], who tried to construct a unified theory of [[biology]] and [[physics]]. [[Buckminster Fuller]] tried to popularize this usage, but ''negentropy'' remains common.

In a note to ''[[What is Life?]]'' Schrödinger explained his use of this phrase.
{{cquote|... if I had been catering for them [physicists] alone I should have let the discussion turn on ''[[Thermodynamic free energy|free energy]]'' instead. It is the more familiar notion in this context. But this highly technical term seemed linguistically too near to ''[[energy]]'' for making the average reader alive to the contrast between the two things.}}

==Information theory==
{{Missing information|section|the mathematical treatment of negentropy in information theory|date=December 2024|talksection=Section on information theory is deficient}}
In [[information theory]] and [[statistics]], negentropy is used as a measure of distance to normality.<ref>Aapo Hyvärinen, [http://www.cis.hut.fi/aapo/papers/NCS99web/node32.html Survey on Independent Component Analysis, node32: Negentropy], Heli University of Technology Laboratory of Computer and Information Science</ref><ref>Aapo Hyvärinen and Erkki Oja, [http://www.cis.hut.fi/aapo/papers/IJCNN99_tutorialweb/node14.html Independent Component Analysis: A Tutorial, node14: Negentropy], Helsinki University of Technology Laboratory of Computer and Information Science</ref><ref>Ruye Wang, [http://fourier.eng.hmc.edu/e161/lectures/ica/node4.html Independent Component Analysis, node4: Measures of Non-Gaussianity]</ref> Out of all [[Distribution (mathematics)|distributions]] with a given mean and variance, the normal or [[Gaussian distribution]] is the one with the highest [[entropy]]. Negentropy measures the difference in entropy between a given distribution and the Gaussian distribution with the same mean and variance. Thus, negentropy is always nonnegative, is invariant by any linear invertible change of coordinates, and vanishes [[if and only if]] the signal is Gaussian.

Negentropy is defined as

:<math>J(p_x) = S(\varphi_x) - S(p_x)\,</math>

where <math>S(\varphi_x)</math> is the [[differential entropy]] of the Gaussian density with the same [[mean]] and [[variance]] as <math>p_x</math> and <math>S(p_x)</math> is the differential entropy of <math>p_x</math>:

:<math>S(p_x) = - \int p_x(u) \log p_x(u) \, du</math>

Negentropy is used in [[statistics]] and [[signal processing]]. It is related to network [[Information entropy|entropy]], which is used in [[independent component analysis]].<ref>P. Comon, Independent Component Analysis – a new concept?, ''Signal Processing'', '''36''' 287–314, 1994.</ref><ref>Didier G. Leibovici and Christian Beckmann, [http://www.fmrib.ox.ac.uk/analysis/techrep/tr01dl1/tr01dl1/tr01dl1.html An introduction to Multiway Methods for Multi-Subject fMRI experiment], FMRIB Technical Report 2001, Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB), Department of Clinical Neurology, University of Oxford, John Radcliffe Hospital, Headley Way, Headington, Oxford, UK.</ref>

The negentropy of a distribution is equal to the [[Kullback–Leibler divergence]] between <math>p_x</math> and a Gaussian distribution with the same mean and variance as <math>p_x</math> (see ''{{section link|Differential entropy#Maximization in the normal distribution}}'' for a proof). In particular, it is always nonnegative.

==Correlation between statistical negentropy and Gibbs' free energy==
[[File:Wykres Gibbsa.svg|275px|thumb|right|[[Willard Gibbs]]’ 1873 '''available energy''' ([[Thermodynamic free energy|free energy]]) graph, which shows a plane perpendicular to the axis of ''v'' ([[volume]]) and passing through point A, which represents the initial state of the body. MN is the section of the surface of [[dissipated energy]]. Qε and Qη are sections of the planes ''η'' = 0 and ''ε'' = 0, and therefore parallel to the axes of ε ([[internal energy]]) and η ([[entropy]]) respectively. AD and AE are the energy and entropy of the body in its initial state, AB and AC its ''available energy'' ([[Gibbs energy]]) and its ''capacity for entropy'' (the amount by which the entropy of the body can be increased without changing the energy of the body or increasing its volume) respectively.]]
There is a physical quantity closely linked to [[Thermodynamic free energy|free energy]] ([[free enthalpy]]), with a unit of entropy and isomorphic to negentropy known in statistics and information theory. In 1873, [[Josiah Willard Gibbs|Willard Gibbs]] created a diagram illustrating the concept of free energy corresponding to [[free enthalpy]]. On the diagram one can see the quantity called [[capacity for entropy]]. This quantity is the amount of entropy that may be increased without changing an internal energy or increasing its volume.<ref>Willard Gibbs, [http://www.ufn.ru/ufn39/ufn39_4/Russian/r394b.pdf A Method of Geometrical Representation of the Thermodynamic Properties of Substances by Means of Surfaces], ''Transactions of the Connecticut Academy'', 382–404 (1873)</ref> In other words, it is a difference between maximum possible, under assumed conditions, entropy and its actual entropy. It corresponds exactly to the definition of negentropy adopted in statistics and information theory. A similar physical quantity was introduced in 1869 by [[François Jacques Dominique Massieu|Massieu]] for the [[isothermal process]]<ref>Massieu, M. F. (1869a). Sur les fonctions caractéristiques des divers fluides. ''C. R. Acad. Sci.'' LXIX:858–862.</ref><ref>Massieu, M. F. (1869b). Addition au precedent memoire sur les fonctions caractéristiques. ''C. R. Acad. Sci.'' LXIX:1057–1061.</ref><ref>Massieu, M. F. (1869), ''Compt. Rend.'' '''69''' (858): 1057.</ref> (both quantities differs just with a figure sign) and by then [[Max Planck|Planck]] for the [[Isothermal process|isothermal]]-[[Isobaric process|isobaric]] process.<ref>Planck, M. (1945). ''Treatise on Thermodynamics''. Dover, New York.</ref> More recently, the Massieu–Planck [[thermodynamic potential]], known also as ''[[free entropy]]'', has been shown to play a great role in the so-called entropic formulation of [[statistical mechanics]],<ref>Antoni Planes, Eduard Vives, [http://www.ecm.ub.es/condensed/eduard/papers/massieu/node2.html Entropic Formulation of Statistical Mechanics] {{Webarchive|url=https://web.archive.org/web/20081011011717/http://www.ecm.ub.es/condensed/eduard/papers/massieu/node2.html |date=2008-10-11 }}, Entropic variables and Massieu–Planck functions 2000-10-24 Universitat de Barcelona</ref> applied among the others in molecular biology<ref>John A. Scheilman, [http://www.biophysj.org/cgi/reprint/73/6/2960.pdf Temperature, Stability, and the Hydrophobic Interaction], ''Biophysical Journal'' '''73''' (December 1997), 2960–2964, Institute of Molecular Biology, University of Oregon, Eugene, Oregon 97403 USA</ref> and thermodynamic non-equilibrium processes.<ref>Z. Hens and X. de Hemptinne, [https://arxiv.org/abs/chao-dyn/9604008 Non-equilibrium Thermodynamics approach to Transport Processes in Gas Mixtures], Department of Chemistry, Catholic University of Leuven, Celestijnenlaan 200 F, B-3001 Heverlee, Belgium</ref>

:: <math>J = S_\max - S = -\Phi = -k \ln Z\,</math>

::where:
::<math>S</math> is [[entropy]]
::<math>J</math> is negentropy (Gibbs "capacity for entropy")
::<math>\Phi</math> is the [[Free entropy|Massieu potential]]
::<math>Z</math> is the [[Partition function (statistical mechanics)|partition function]]
::<math>k</math> the [[Boltzmann constant]]

In particular, mathematically the negentropy (the negative entropy function, in physics interpreted as free entropy) is the [[convex conjugate]] of [[LogSumExp]] (in physics interpreted as the free energy).

==Brillouin's negentropy principle of information==
In 1953, [[Léon Brillouin]] derived a general equation<ref>Leon Brillouin, The negentropy principle of information, ''J. Applied Physics'' '''24''', 1152–1163 1953</ref> stating that the changing of an information bit value requires at least <math>kT\ln 2</math> energy. This is the same energy as the work [[Leó Szilárd]]'s engine produces in the idealistic case. In his book,<ref>Leon Brillouin, ''Science and Information theory'', Dover, 1956</ref> he further explored this problem concluding that any cause of this bit value change (measurement, decision about a yes/no question, erasure, display, etc.) will require the same amount of energy.

==See also==
* [[Exergy]]
* [[Free entropy]]
* [[Entropy in thermodynamics and information theory]]

==Notes==
{{Reflist|20em}}
{{Wiktionary}}

[[Category:Entropy and information]]
[[Category:Statistical deviation and dispersion]]
[[Category:Thermodynamic entropy]]