View source for Hopfield network

{{short description|Form of artificial neural network}}
A '''Hopfield network''' (or '''associative memory''') is a form of [[recurrent neural network]], or a [[spin glass]] system, that can serve as a [[content-addressable memory]]. The Hopfield network, named for [[John Hopfield]], consists of a single layer of neurons, where each neuron is connected to every other neuron except itself. These connections are bidirectional and symmetric, meaning the weight of the connection from neuron ''i'' to neuron ''j'' is the same as the weight from neuron ''j'' to neuron ''i''. Patterns are associatively recalled by fixing certain inputs, and dynamically evolve the network to minimize an energy function, towards local energy minimum states that correspond to stored patterns. Patterns are associatively learned (or "stored") by a [[Hebbian theory|Hebbian learning]] algorithm.

One of the key features of Hopfield networks is their ability to recover complete patterns from partial or noisy inputs, making them robust in the face of incomplete or corrupted data. Their connection to statistical mechanics, recurrent networks, and human cognitive psychology has led to their application in various fields, including [[physics]], [[psychology]], [[neuroscience]], and machine learning theory and practice.

== History ==
{{Primary sources|date=October 2024}}
One origin of associative memory is human [[cognitive psychology]], specifically the [[Associative memory (psychology)|associative memory]]. [[Frank Rosenblatt]] studied "close-loop cross-coupled perceptrons", which are 3-layered [[perceptron]] networks whose middle layer contains recurrent connections that change by a [[Hebbian theory|Hebbian learning]] rule.<ref>F. Rosenblatt, "[[iarchive:SelfOrganizingSystems/page/n87/mode/1up|Perceptual Generalization over Transformation Groups]]", pp. 63--100 in ''Self-organizing Systems: Proceedings of an Inter-disciplinary Conference, 5 and 6 May 1959''. Edited by Marshall C. Yovitz and Scott Cameron. London, New York, [etc.], Pergamon Press, 1960. ix, 322 p.</ref>{{Pg|pages=73-75}}<ref name=":12">{{Cite book |last=Rosenblatt |first=Frank |url=https://archive.org/details/DTIC_AD0256582/page/n3/mode/2up |title=DTIC AD0256582: PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS |date=1961-03-15 |publisher=Defense Technical Information Center |language=english}}</ref>{{Pg|location=Chapter 19, 21}}

Another model of associative memory is where the output does not loop back to the input. W. K. Taylor proposed such a model trained by Hebbian learning in 1956.<ref>W. K. Taylor, 1956. ''Electrical simulation of some nervous system functional activities''. Information Theory 3, E. C. Cherry (ed.), pp. 314-328. London: Butterworths.</ref> [[Karl Steinbuch]], who wanted to understand learning, and inspired by watching his children learn,<ref>''[https://www.itiv.kit.edu/downloads/euology_1_.pdf Eulogy: 1917 Karl Steinbuch 2005]'', by Bernard Widrow, Reiner Hartenstein, Robert Hecht-Nielsen, IEEE Computational Intelligence Society. page 5. August 2005.</ref> published the [[Lernmatrix]] in 1961.<ref>{{Cite journal |last=Steinbuch |first=K. |date=1961-01-01 |title=Die Lernmatrix |url=https://link.springer.com/article/10.1007/BF00293853 |journal=Kybernetik |language=de |volume=1 |issue=1 |pages=36–45 |doi=10.1007/BF00293853 |issn=1432-0770}}</ref><ref>{{Cite book |last=Steinbuch |first=Karl |url=https://openlibrary.org/books/OL27019478M/Automat_und_Mensch |title=Automat und Mensch: über menschliche und maschinelle Intelligenz |date=1961 |publisher=Springer |isbn=978-3-642-53168-2 |location=Berlin|ol=27019478M }}</ref> It was translated to English in 1963.<ref>{{Cite journal |last1=Steinbuch |first1=K. |last2=Piske |first2=U. A. W. |date=December 1963 |title=Learning matrices and their applications |url=https://ieeexplore.ieee.org/document/4038032 |journal=IEEE Transactions on Electronic Computers |volume=EC-12 |issue=6 |pages=846–862 |doi=10.1109/PGEC.1963.263588 |issn=0367-7508}}</ref> Similar research was done with the ''correlogram'' of D. J. Willshaw et al. in 1969.<ref>{{Cite journal |last1=Willshaw |first1=D. J. |last2=Buneman |first2=O. P. |last3=Longuet-Higgins |first3=H. C. |date=June 1969 |title=Non-Holographic Associative Memory |url=https://www.nature.com/articles/222960a0 |journal=Nature |volume=222 |issue=5197 |pages=960–962 |doi=10.1038/222960a0 |pmid=5789326 |bibcode=1969Natur.222..960W |issn=0028-0836}}</ref> [[Teuvo Kohonen]] trained an associative memory by gradient descent in 1974.<ref>{{Cite journal |last=Kohonen |first=T. |date=April 1974 |title=An Adaptive Associative Memory Principle |url=https://ieeexplore.ieee.org/document/1672553 |journal=IEEE Transactions on Computers |volume=C-23 |issue=4 |pages=444–445 |doi=10.1109/T-C.1974.223960 |issn=0018-9340}}</ref>[[File:Typical_connections_in_a_close-loop_cross-coupled_perceptron.png|thumb|A close-loop cross-coupled perceptron network. ''Principles of Neurodynamics'' (1961){{Pg|page=403|location=Fig. 47}}.]]
Another origin of associative memory was [[statistical mechanics]]. The Ising model was published in 1920s as a model of magnetism, however it studied the thermal equilibrium, which does not change with time. [[Roy J. Glauber]] in 1963 studied the Ising model evolving in time, as a process towards thermal equilibrium ([[Glauber dynamics]]), adding in the component of time.<ref name=":22">{{cite journal |last1=Glauber |first1=Roy J. |date=February 1963 |title=Roy J. Glauber "Time-Dependent Statistics of the Ising Model" |url=https://aip.scitation.org/doi/abs/10.1063/1.1703954 |journal=Journal of Mathematical Physics |volume=4 |issue=2 |pages=294–307 |doi=10.1063/1.1703954 |access-date=2021-03-21}}</ref>

The second component to be added was adaptation to stimulus. Described independently by Kaoru Nakano in 1971<ref name="Nakano1971">{{cite book |last1=Nakano |first1=Kaoru |title=Pattern Recognition and Machine Learning |date=1971 |isbn=978-1-4615-7568-9 |pages=172–186 |chapter=Learning Process in a Model of Associative Memory |doi=10.1007/978-1-4615-7566-5_15}}</ref><ref name="Nakano1972">{{cite journal |last1=Nakano |first1=Kaoru |date=1972 |title=Associatron-A Model of Associative Memory |journal=IEEE Transactions on Systems, Man, and Cybernetics |volume=SMC-2 |issue=3 |pages=380–388 |doi=10.1109/TSMC.1972.4309133}}</ref> and [[Shun'ichi Amari]] in 1972,<ref name="Amari1972">{{cite journal |last1=Amari |first1=Shun-Ichi |date=1972 |title=Learning patterns and pattern sequences by self-organizing nets of threshold elements |journal=IEEE Transactions |volume=C |issue=21 |pages=1197–1206}}</ref> they proposed to modify the weights of an Ising model by [[Hebbian theory|Hebbian learning]] rule as a model of associative memory. The same idea was published by {{ill|William A. Little (physicist)|lt=William A. Little|de|William A. Little}} in 1974,<ref name="little74">{{cite journal |last=Little |first=W. A. |year=1974 |title=The Existence of Persistent States in the Brain |journal=Mathematical Biosciences |volume=19 |issue=1–2 |pages=101–120 |doi=10.1016/0025-5564(74)90031-5}}</ref> who was acknowledged by Hopfield in his 1982 paper.

See Carpenter (1989)<ref>{{Cite journal |last=Carpenter |first=Gail A |date=1989-01-01 |title=Neural network models for pattern recognition and associative memory |url=https://www.sciencedirect.com/science/article/abs/pii/089360808990035X |journal=Neural Networks |volume=2 |issue=4 |pages=243–257 |doi=10.1016/0893-6080(89)90035-X |issn=0893-6080}}</ref> and Cowan (1990)<ref>{{Cite journal |last=Cowan |first=Jack D. |date=January 1990 |title=Discussion: McCulloch-Pitts and related neural nets from 1943 to 1989 |url=http://link.springer.com/10.1007/BF02459569 |journal=Bulletin of Mathematical Biology |language=en |volume=52 |issue=1–2 |pages=73–97 |doi=10.1007/BF02459569 |issn=0092-8240}}</ref> for a technical description of some of these early works in associative memory.

The [[Spin glass#Sherrington–Kirkpatrick model|Sherrington–Kirkpatrick model]] of spin glass, published in 1975,<ref>{{Cite journal |last1=Sherrington |first1=David |last2=Kirkpatrick |first2=Scott |date=1975-12-29 |title=Solvable Model of a Spin-Glass |url=https://link.aps.org/doi/10.1103/PhysRevLett.35.1792 |journal=Physical Review Letters |volume=35 |issue=26 |pages=1792–1796 |doi=10.1103/PhysRevLett.35.1792 |bibcode=1975PhRvL..35.1792S |issn=0031-9007}}</ref> is the Hopfield network with random initialization. Sherrington and Kirkpatrick found that it is highly likely for the energy function of the SK model to have many local minima. In the 1982 paper, Hopfield applied this recently developed theory to study the Hopfield network with binary activation functions.<ref name="Hopfield1982">{{cite journal |last1=Hopfield |first1=J. J. |date=1982 |title=Neural networks and physical systems with emergent collective computational abilities |journal=Proceedings of the National Academy of Sciences |volume=79 |issue=8 |pages=2554–2558 |bibcode=1982PNAS...79.2554H |doi=10.1073/pnas.79.8.2554 |pmc=346238 |pmid=6953413 |doi-access=free}}</ref> In a 1984 paper he extended this to continuous activation functions.<ref name=":0" /> It became a standard model for the study of neural networks through statistical mechanics.<ref>{{Cite book |last1=Engel |first1=A. |title=Statistical mechanics of learning |last2=Broeck |first2=C. van den |date=2001 |publisher=Cambridge University Press |isbn=978-0-521-77307-2 |location=Cambridge, UK; New York, NY}}</ref><ref>{{Cite journal |last1=Seung |first1=H. S. |last2=Sompolinsky |first2=H. |last3=Tishby |first3=N. |date=1992-04-01 |title=Statistical mechanics of learning from examples |url=https://journals.aps.org/pra/abstract/10.1103/PhysRevA.45.6056 |journal=Physical Review A |volume=45 |issue=8 |pages=6056–6091 |doi=10.1103/PhysRevA.45.6056|pmid=9907706 |bibcode=1992PhRvA..45.6056S }}</ref>

A major advance in memory storage capacity was developed by Dimitry Krotov and Hopfield in 2016<ref name=":1" /> through a change in network dynamics and energy function. This idea was further extended by Demircigil and collaborators in 2017.<ref name=":2" /> The continuous dynamics of large memory capacity models was developed in a series of papers between 2016 and 2020.<ref name=":1" /><ref name=":3" /><ref name=":4" /> Large memory storage capacity Hopfield Networks are now called Dense Associative Memories or [[modern Hopfield network]]s.

In 2024, John J. Hopfield and [[Geoffrey E. Hinton]] were awarded the [[Nobel Prize in Physics]] for their foundational contributions to machine learning, such as the Hopfield network.

==Structure==
[[Image:Hopfield-net-vector.svg|thumb|A Hopfield net with four units]]
The units in Hopfield nets are binary threshold units, i.e. the units only take on two different values for their states, and the value is determined by whether or not the unit's input exceeds its threshold <math> U_i </math>. Discrete Hopfield nets describe relationships between binary (firing or not-firing) neurons <math>1,2,\ldots,i,j,\ldots,N</math>.<ref name="Hopfield1982" />  At a certain time, the state of the neural net is described by a vector <math> V </math>, which records which neurons are firing in a binary word of <math> N </math> bits.

The interactions <math> w_{ij} </math> between neurons have units that usually take on values of 1 or −1, and this convention will be used throughout this article. However, other literature might use units that take values of 0 and 1.  These interactions are "learned" via [[Hebbian theory|Hebb's law of association]], such that, for a certain state  <math> V^s </math> and distinct nodes <math>i,j</math>

<math> w_{ij} = V_i^s V_j^s </math>

but <math> w_{ii} = 0 </math>.

(Note that the Hebbian learning rule takes the form <math> w_{ij} = (2V_i^s - 1)(2V_j^s -1) </math> when the units assume values in <math> \{0, 1\} </math>.)

Once the network is trained, <math> w_{ij} </math> no longer evolve.  If a new state of neurons <math> V^{s'} </math> is introduced to the neural network, the net acts on neurons such that

* <math>V^{s'}_i \rightarrow 1 </math> if <math> \sum_j w_{ij} V^{s'}_j > U_i </math>
* <math>V^{s'}_i \rightarrow -1 </math> if <math> \sum_j w_{ij} V^{s'}_j < U_i </math>

where <math>U_i</math> is the threshold value of the i'th neuron (often taken to be 0).<ref>{{cite journal |last1=Hopfield |first1=J. J. |title=Neural networks and physical systems with emergent collective computational abilities |journal= Proceedings of the National Academy of Sciences|date= 1982 |volume=79 |issue=8 |pages=2554–2558 |doi=10.1073/pnas.79.8.2554 |pmid=6953413 |pmc=346238 |bibcode=1982PNAS...79.2554H |doi-access=free }}</ref> In this way, Hopfield networks have the ability to "remember" states stored in the interaction matrix, because if a new state <math>V^{s'} </math> is subjected to the interaction matrix, each neuron will change until it matches the original state <math>V^{s} </math> (see the Updates section below).

The connections in a Hopfield net typically have the following restrictions:
* <math>w_{ii}=0, \forall i</math> (no unit has a connection with itself)
* <math>w_{ij} = w_{ji}, \forall i,j</math> (connections are symmetric)

The constraint that weights are symmetric guarantees that the energy function decreases monotonically while following the activation rules.<ref>{{cite book|last=MacKay|first=David J. C.|author-link=David J.C. MacKay|title=Information Theory, Inference and Learning Algorithms|url=https://archive.org/details/informationtheor00mack_607|url-access=limited|chapter=42. Hopfield Networks|year=2003|publisher=[[Cambridge University Press]]|page=[https://archive.org/details/informationtheor00mack_607/page/n519 508]|quote=This convergence proof depends crucially on the fact that the Hopfield network's connections are ''symmetric''. It also depends on the updates being made asynchronously.|isbn=978-0521642989}}</ref> A network with asymmetric weights may exhibit some periodic or chaotic behaviour; however, Hopfield found that this behavior is confined to relatively small parts of the phase space and does not impair the network's ability to act as a content-addressable associative memory system.

Hopfield also modeled neural nets for continuous values, in which the electric output of each neuron is not binary but some value between 0 and 1.<ref name=":0">{{cite journal |last1=Hopfield |first1=J. J. |title=Neurons with graded response have collective computational properties like those of two-state neurons |journal= Proceedings of the National Academy of Sciences|date= 1984 |volume=81 |issue=10 |pages=3088–3092 |doi=10.1073/pnas.81.10.3088 |pmid=6587342 |pmc=345226 |bibcode=1984PNAS...81.3088H |doi-access=free }}</ref>  He found that this type of network was also able to store and reproduce memorized states.

Notice that every pair of units ''i'' and ''j'' in a Hopfield network has a connection that is described by the connectivity weight <math> w_{ij} </math>. In this sense, the Hopfield network can be formally described as a complete undirected graph <math> G = \langle V, f\rangle </math>, where <math>V</math> is a set of [[Artificial neuron|McCulloch–Pitts neurons]] and <math>f:V^2 \rightarrow \mathbb R</math> is a function that links pairs of units to a real value, the connectivity weight.

==Updating==

Updating one unit (node in the graph simulating the artificial neuron) in the Hopfield network is performed using the following rule:

<math>s_i \leftarrow \left\{\begin{array}{ll} +1 & \text{if }\sum_{j}{w_{ij}s_j}\geq\theta_i, \\
 -1 & \text{otherwise.}\end{array}\right.</math>

where:
* <math>w_{ij}</math> is the strength of the connection weight from unit j to unit i (the weight of the connection).
* <math>s_i</math> is the state of unit i.
* <math>\theta_i</math> is the threshold of unit i.

Updates in the Hopfield network can be performed in two different ways:
* '''Asynchronous''': Only one unit is updated at a time. This unit can be picked at random, or a pre-defined order can be imposed from the very beginning.
* '''Synchronous''': All units are updated at the same time. This requires a central clock to the system in order to maintain synchronization. This method is viewed by some as less realistic, based on an absence of observed global clock influencing analogous biological or physical systems of interest.

===Neurons "attract or repel each other" in state space ===
The weight between two units has a powerful impact upon the values of the neurons. Consider the connection weight <math>w_{ij}</math> between two neurons i and j. If <math>w_{ij} > 0 </math>, the updating rule implies that:
* when <math>s_j = 1</math>, the contribution of ''j'' in the weighted sum is positive. Thus, <math>s_{i}</math> is pulled by ''j'' towards its value <math>s_{i} = 1</math>
* when <math>s_j = -1</math>, the contribution of ''j'' in the weighted sum is negative. Then again, <math>s_i</math> is pushed by ''j'' towards its value <math>s_i = -1</math>

Thus, the values of neurons ''i'' and ''j'' will converge if the weight between them is positive. Similarly, they will diverge if the weight is negative.

== Convergence properties of discrete and continuous Hopfield networks  ==

[https://www.paradise.caltech.edu/bruck.html Bruck] in his paper in 1990<ref name="Bruck1990">{{cite journal |last=Bruck |first=J. |date=October 1990 |title=On the convergence properties of the Hopfield model |url=https://resolver.caltech.edu/CaltechAUTHORS:20120426-132042598 |journal=Proc. IEEE |volume=78 |issue=10 |pages=1579–85 |doi=10.1109/5.58341}}</ref>  studied discrete Hopfield networks and proved a generalized convergence theorem that is based on the connection between the network's dynamics and cuts in the associated graph. This generalization covered both asynchronous as well as synchronous dynamics and presented elementary proofs based on greedy algorithms for max-cut in graphs. A subsequent paper<ref name="Uykan2019">{{cite journal |last=Uykan |first=Z. |date=September 2020 |title=On the Working Principle of the Hopfield Neural Networks and its Equivalence to the GADIA in Optimization |url=https://ieeexplore.ieee.org/document/8859641 |journal=IEEE Transactions on Neural Networks and Learning Systems |volume=31 |issue=9 |pages=3294–3304 |doi=10.1109/TNNLS.2019.2940920 |pmid=31603804 |s2cid=204331533}}</ref> further investigated the behavior of any neuron in both discrete-time and continuous-time Hopfield networks when the corresponding energy function is minimized during an optimization process.  Bruck showed<ref name="Bruck1990"/> that neuron ''j'' changes its state ''if and only if'' it further decreases the following biased pseudo-cut.  The discrete Hopfield network minimizes the following biased pseudo-cut<ref name="Uykan2019"/> for the synaptic weight matrix of the Hopfield net.

<math> J_{pseudo-cut}(k) = 
\sum_{i \in C_1(k)} \sum_{j \in C_2(k)} w_{ij} + \sum_{j \in C_1(k)} {\theta_j} </math>

where <math> C_1(k) </math> and <math> C_2(k) </math> represents the set of neurons which are −1 and +1, respectively, at time <math> k </math>.  For further details, see the recent paper.<ref name="Uykan2019"/>

The discrete-time Hopfield Network always minimizes exactly the following pseudo-cut<ref name="Bruck1990" /><ref name="Uykan2019" />

: <math> U(k) = \sum_{i=1}^N \sum_{j=1}^{N} w_{ij} ( s_i(k) - s_j(k) )^2 + 2 \sum_{j=1}^N \theta_j s_j(k) </math>

The continuous-time Hopfield network always minimizes an upper bound to the following weighted cut<ref name="Uykan2019" />

: <math> V(t) = \sum_{i=1}^N \sum_{j=1}^N w_{ij} ( f(s_i(t)) - f(s_j(t) )^2 + 2 \sum_{j=1}^N \theta_j f(s_j(t)) </math>

where <math> f(\cdot) </math> is a zero-centered sigmoid function.

The complex Hopfield network, on the other hand, generally tends to minimize the so-called shadow-cut of the complex weight matrix of the net.<ref name="Uykan2020">{{cite journal |first=Z. |last=Uykan |title=Shadow-Cuts Minimization/Maximization and Complex Hopfield Neural Networks |journal=IEEE Transactions on Neural Networks and Learning Systems |volume=32 |issue=3 |pages=1096–1109 |date=March 2021 |doi=10.1109/TNNLS.2020.2980237 |pmid=32310787 |s2cid=216047831 |doi-access=free }}</ref>

==Energy==

[[File:Energy landscape.png|thumb|right|500px|Energy Landscape of a Hopfield Network, highlighting the current state of the network (up the hill), an attractor state to which it will eventually converge, a minimum energy level and a basin of attraction shaded in green. Note how the update of the Hopfield Network is always going down in Energy.]]

Hopfield nets have a scalar value associated with each state of the network, referred to as the "energy", ''E'', of the network, where:
:<math>E = -\frac12\sum_{i,j} w_{ij} s_i s_j -\sum_i \theta_i s_i</math>

This quantity is called "energy" because it either decreases or stays the same upon network units being updated. Furthermore, under repeated updating the network will eventually converge to a state which is a [[local minimum]] in the energy function (which is considered to be a [[Lyapunov function]]).<ref name="Hopfield1982" /> Thus, if a state is a local minimum in the energy function it is a stable state for the network. Note that this energy function belongs to a general class of models in [[physics]] under the name of [[Ising model]]s; these in turn are a special case of [[Markov networks]], since the associated [[probability measure]], the [[Gibbs measure]], has the [[Markov property]].

== Hopfield network in optimization ==

[[John Hopfield|Hopfield]] and [[David Tank|Tank]] presented the Hopfield network application in solving the classical traveling-salesman problem in 1985.<ref name="hopfieldtank85">{{cite journal |first1=J.J. |last1=Hopfield |first2=D.W. |last2=Tank |title=Neural computation of decisions in optimization problems |journal=Biological Cybernetics |volume=52 |pages=141–6 |year=1985 |issue=3 |doi=10.1007/BF00339943 |pmid=4027280 |s2cid=36483354 |url=}}</ref> Since then, the Hopfield network has been widely used for optimization. The idea of using the Hopfield network in optimization problems is straightforward: If a constrained/unconstrained cost function can be written in the form of the Hopfield energy function E, then there exists a Hopfield network whose equilibrium points represent solutions to the constrained/unconstrained optimization problem.  Minimizing the Hopfield energy function both minimizes the objective function and satisfies the constraints also as the constraints are "embedded" into the synaptic weights of the network. Although including the optimization constraints into the synaptic weights in the best possible way is a challenging task, many difficult optimization problems with constraints in different disciplines have been converted to the Hopfield energy function: Associative memory systems, Analog-to-Digital conversion, job-shop scheduling problem, quadratic assignment and other related NP-complete problems, channel allocation problem in wireless networks, mobile ad-hoc network routing problem, image restoration, system identification, combinatorial optimization, etc, just to name a few. However, while it is possible to convert hard optimization problems to Hopfield energy functions, it does not guarantee convergence to a solution (even in exponential time).<ref>{{Cite journal |last1=Bruck |first1=Jehoshua |last2=Goodman |first2=Joseph W |date=1990-06-01 |title=On the power of neural networks for solving hard problems |url=https://linkinghub.elsevier.com/retrieve/pii/0885064X9090001T |journal=Journal of Complexity |volume=6 |issue=2 |pages=129–135 |doi=10.1016/0885-064X(90)90001-T |issn=0885-064X}}</ref>

== Initialization and running ==

Initialization of the Hopfield networks is done by setting the values of the units to the desired start pattern. Repeated updates are then performed until the network converges to an attractor pattern. Convergence is generally assured, as Hopfield proved that the attractors of this [[nonlinear dynamical system]] are stable, not periodic or chaotic as in some other systems.{{citation needed|date=July 2019}} Therefore, in the context of Hopfield networks, an attractor pattern is a final stable state, a pattern that cannot change any value within it under updating.{{citation needed|date=July 2019}}

==Training==
Training a Hopfield net involves lowering the energy of states that the net should "remember". This allows the net to serve as a content addressable memory system, that is to say, the network will converge to a "remembered" state if it is given only part of the state. The net can be used to recover from a distorted input to the trained state that is most similar to that input. This is called associative memory because it recovers memories on the basis of similarity. For example, if we train a Hopfield net with five units so that the state (1, −1, 1, −1, 1) is an energy minimum, and we give the network the state (1, −1, −1, −1, 1) it will converge to (1, −1, 1, −1, 1). Thus, the network is properly trained when the energy of states which the network should remember are local minima. Note that, in contrast to [[Perceptron#Learning algorithm|Perceptron]] training, the thresholds of the neurons are never updated.

===Learning rules===
There are various different [[learning rule]]s that can be used to store information in the memory of the Hopfield network. It is desirable for a learning rule to have both of the following two properties:
* ''Local'': A learning rule is ''local'' if each weight is updated using information available to neurons on either side of the connection that is associated with that particular weight.
* ''Incremental'': New patterns can be learned without using information from the old patterns that have been also used for training. That is, when a new pattern is used for training, the new values for the weights only depend on the old values and on the new pattern.<ref name="storkey1991basins" />

These properties are desirable, since a learning rule satisfying them is more biologically plausible. For example, since the human brain is always learning new concepts, one can reason that human learning is incremental. A learning system that was not incremental would generally be trained only once, with a huge batch of training data.

===Hebbian learning rule for Hopfield networks===
[[Hebbian theory]] was introduced by Donald Hebb in 1949 in order to explain "associative learning", in which simultaneous activation of neuron cells leads to pronounced increases in synaptic strength between those cells.<ref>{{harvnb|Hebb|1949}}</ref> It is often summarized as "Neurons that fire together wire together. Neurons that fire out of sync fail to link".

The Hebbian rule is both local and incremental. For the Hopfield networks, it is implemented in the following manner when learning <math>n</math>
binary patterns:

<math> w_{ij}=\frac{1}{n}\sum_{\mu=1}^{n}\epsilon_{i}^\mu \epsilon_{j}^\mu </math>

where <math>\epsilon_i^\mu</math> represents bit i from pattern <math>\mu</math>.

If the bits corresponding to neurons i and j are equal in pattern <math>\mu</math>, then the product  <math> \epsilon_{i}^\mu \epsilon_{j}^\mu </math> will be positive. This would, in turn, have a positive effect on the weight <math>w_{ij} </math> and the values of i and j will tend to become equal. The opposite happens if the bits corresponding to neurons i and j are different.

===Storkey learning rule===
This rule was introduced by [[Amos Storkey]] in 1997 and is both local and incremental. Storkey also showed that a Hopfield network trained using this rule has a greater capacity than a corresponding network trained using the Hebbian rule.<ref name="storkey1997">{{cite book |last=Storkey |first=Amos |chapter=Increasing the capacity of a Hopfield network without sacrificing functionality |title=Artificial Neural Networks – ICANN'97 |year=1997 |citeseerx=10.1.1.33.103 |pages=451–6 |doi=10.1007/BFb0020196 |publisher=Springer |series= Lecture Notes in Computer Science |volume=1327 |isbn=978-3-540-69620-9}}</ref> The weight matrix of an attractor neural network{{clarify|reason=What is an attractor NN?|date=July 2019}} is said to follow the Storkey learning rule if it obeys:

<math> w_{ij}^{\nu} = w_{ij}^{\nu-1}
		    +\frac{1}{n}\epsilon_{i}^{\nu} \epsilon_{j}^{\nu} 
		    -\frac{1}{n}\epsilon_{i}^{\nu} h_{ji}^{\nu}
		    -\frac{1}{n}\epsilon_{j}^{\nu} h_{ij}^{\nu}
		    </math>

where <math> h_{ij}^{\nu} = \sum_{k=1~:~i\neq k\neq j}^{n} w_{ik}^{\nu-1}\epsilon_{k}^{\nu} </math> is a form of ''local field''<ref name="storkey1991basins">{{cite journal |last1=Storkey |first1=A.J. |first2=R. |last2=Valabregue |title=The basins of attraction of a new Hopfield learning rule |journal=Neural Networks |volume=12 |issue=6 |pages=869–876 |year=1999 |doi=10.1016/S0893-6080(99)00038-6 |pmid=12662662 |citeseerx=10.1.1.19.4681}}</ref> at neuron i.
		    
This learning rule is local, since the synapses take into account only neurons at their sides. The rule makes use of more information from the patterns and weights than the generalized Hebbian rule, due to the effect of the local field.

==Spurious patterns==
{{Double image
| direction         = horizontal
| image1            = Hopfield_dynamics.gif
| image2            = Hopfield_dynamics_spurious.gif
| caption1          = 10 random bits initially flipped
| caption2          = 15 random bits initially flipped (converging to the spurious pattern)
| footer            = Hopfield dynamics of a network of 5x5=25 neurons trained to store a single pattern "E" in black
}}
Patterns that the network uses for training (called ''retrieval states'') become attractors of the system. Repeated updates would eventually lead to convergence to one of the retrieval states. However, sometimes the network will converge to spurious patterns (different from the training patterns).<ref name="hertz1991neural">{{harvnb|Hertz|1991}}</ref> In fact, the number of spurious patterns can be exponential in the number of stored patterns, even if the stored patterns are orthogonal.<ref>{{Cite journal |last1=Bruck |first1=J. |last2=Roychowdhury |first2=V.P. |date=1990 |title=On the number of spurious memories in the Hopfield model (neural network) |url=https://ieeexplore.ieee.org/document/52486 |journal=IEEE Transactions on Information Theory |volume=36 |issue=2 |pages=393–397 |doi=10.1109/18.52486}}</ref> The energy in these spurious patterns is also a local minimum. For each stored pattern x, the negation -x is also a spurious pattern.

A spurious state can also be a [[linear combination]] of an odd number of retrieval states. For example, when using 3 patterns <math> \mu_1, \mu_2, \mu_3</math>, one can get the following spurious state:

<math> \epsilon_{i}^{\rm{mix}} = \pm \sgn(\pm \epsilon_{i}^{\mu_{1}} 
			         \pm \epsilon_{i}^{\mu_{2}}
			         \pm \epsilon_{i}^{\mu_{3}})
</math>

Spurious patterns that have an even number of states cannot exist, since they might sum up to zero<ref name="hertz1991neural" />

==Capacity==

The Network capacity of the Hopfield network model is determined by neuron amounts and connections within a given network. Therefore, the number of memories that are able to be stored is dependent on neurons and connections. Furthermore, it was shown that the recall accuracy between vectors and nodes was 0.138 (approximately 138 vectors can be recalled from storage for every 1000 nodes) (Hertz et al., 1991). Therefore, it is evident that many mistakes will occur if one tries to store a large number of vectors. When the Hopfield model does not recall the right pattern, it is possible that an intrusion has taken place, since semantically related items tend to confuse the individual, and recollection of the wrong pattern occurs. Therefore, the Hopfield network model is shown to confuse one stored item with that of another upon retrieval. Perfect recalls and high capacity, >0.14, can be loaded in the network by Storkey learning method; ETAM,<ref>{{cite journal | last1=Liou | first1=C.-Y. | last2=Lin  | first2=S.-L. | s2cid=35025761 | title=Finite memory loading in hairy neurons |journal=Natural Computing |volume=5 |issue=1 |pages=15–42 |year=2006 |doi=10.1007/s11047-004-5490-x |url=http://ntur.lib.ntu.edu.tw//bitstream/246246/155192/1/19.pdf }}</ref><ref>{{cite journal | last1=Liou | first1=C.-Y. | last2=Yuan  | first2=S.-K. | s2cid=6168346 | title=Error Tolerant Associative Memory |journal=Biological Cybernetics |volume=81 | issue=4 | pages=331–342 |year=1999 |doi=10.1007/s004220050566 | pmid=10541936 }}</ref> ETAM experiments also in.<ref>{{cite thesis | last=Yuan | first=S.-K. | title=Expanding basins of attraction of the associative memory | institution=National Taiwan University | degree=Master | id=991010725609704786 | date=June 1997 | url=https://ntu.primo.exlibrisgroup.com/discovery/fulldisplay?vid=886NTU_INST:886NTU_INST&search_scope=MyInstitution&tab=LibraryCatalog&docid=alma991010725609704786&lang=en&context=L&adaptor=Local%20Search%20Engine&query=any,contains,%E5%8A%89%E9%95%B7%E9%81%A0&facet=rtype,include,manuscripts&facet=searchcreationdate,include,1977%7C,%7C2019&facet=searchcreationdate,include,1992%7C,%7C2010&offset=0}}</ref> Ulterior models inspired by the Hopfield network were later devised to raise the storage limit and reduce the retrieval error rate, with some being capable of [[One-shot learning (computer vision)|one-shot learning]].<ref>{{cite book |last1=ABOUDIB |first1=Ala |last2=GRIPON |first2=Vincent |last3=JIANG |first3=Xiaoran |chapter=A study of retrieval algorithms of sparse messages in networks of neural cliques |title=COGNITIVE 2014 : The 6th International Conference on Advanced Cognitive Technologies and Applications |date=2014 |pages=140–6 |arxiv=1308.4506 |bibcode=2013arXiv1308.4506A |chapter-url=https://hal.archives-ouvertes.fr/hal-01058303/}}</ref>

The storage capacity can be given as <math>C \cong \frac{n}{2\log_2n}</math> where <math>n</math> is the number of neurons in the net.

==Human memory==
The Hopfield network is a model for human associative learning and recall.<ref>{{cite book |last1=Amit |first1=D.J. |url=https://books.google.com/books?id=fvLYch1yQncC |title=Modeling Brain Function: The World of Attractor Neural Networks |date=1992 |publisher=Cambridge University Press |isbn=978-0-521-42124-9}}</ref><ref>{{cite book |last=Rolls |first=Edmund T. |url=https://books.google.com/books?id=on_ADAAAQBAJ |title=Cerebral Cortex: Principles of Operation |publisher=Oxford University Press |year=2016 |isbn=978-0-19-878485-2}}</ref> It accounts for [[Association (psychology)|associative]] [[memory]] through the incorporation of memory vectors. Memory vectors can be slightly used, and this would spark the retrieval of the most similar vector in the network. However, we will find out that due to this process, intrusions can occur. In associative memory for the Hopfield network, there are two types of operations: auto-association and hetero-association. The first being when a vector is associated with itself, and the latter being when two different vectors are associated in storage. Furthermore, both types of operations are possible to store within a single memory matrix, but only if that given representation matrix is not one or the other of the operations, but rather the combination (auto-associative and hetero-associative) of the two.

Hopfield's network model utilizes the same learning rule as [[Hebbian theory|Hebb's (1949) learning rule]], which characterised learning as being a result of the strengthening of the weights in cases of neuronal activity.

Rizzuto and Kahana (2001) were able to show that the neural network model can account for repetition on recall accuracy by incorporating a probabilistic-learning algorithm. During the retrieval process, no learning occurs. As a result, the weights of the network remain fixed, showing that the model is able to switch from a learning stage to a recall stage. By adding contextual drift they were able to show the rapid forgetting that occurs in a Hopfield model during a cued-recall task. The entire network contributes to the change in the activation of any single node.

McCulloch and Pitts' (1943) dynamical rule, which describes the behavior of neurons, does so in a way that shows how the activations of multiple neurons map onto the activation of a new neuron's firing rate, and how the weights of the neurons strengthen the synaptic connections between the new activated neuron (and those that activated it). Hopfield would use McCulloch–Pitts's dynamical rule in order to show how retrieval is possible in the Hopfield network. However, Hopfield would do so in a repetitious fashion. Hopfield would use a nonlinear activation function, instead of using a linear function. This would therefore create the Hopfield dynamical rule and with this, Hopfield was able to show that with the nonlinear activation function, the dynamical rule will always modify the values of the state vector in the direction of one of the stored patterns.

== Dense associative memory or modern Hopfield network ==
{{main|Modern Hopfield network}}
Hopfield networks<ref name="Hopfield1982" /><ref name=":0" /> are recurrent neural networks with dynamical trajectories converging to fixed point attractor states and described by an energy function. The state of each model neuron <math display="inline">i </math> is defined by a time-dependent variable <math>V_i</math>, which can be chosen to be either discrete or continuous. A complete model describes the mathematics of how the future state of activity of each neuron depends on the known present or previous activity of all the neurons.

In the original Hopfield model of associative memory,<ref name="Hopfield1982" /> the variables were binary, and the dynamics were described by a one-at-a-time update of the state of the neurons. An energy function quadratic in the <math>V_i</math> was defined, and the dynamics consisted of changing the activity of each single neuron <math>i</math> only if doing so would lower the total energy of the system. This same idea was extended to the case of <math>V_i</math> being a continuous variable representing the output of neuron <math>i</math>, and <math>V_i</math> being a monotonic function of an input current. The dynamics became expressed as a set of first-order differential equations for which the "energy" of the system always decreased.<ref name=":0" />  The energy in the continuous case has one term which is quadratic in the <math>V_i</math> (as in the binary model), and a second term which depends on the gain function (neuron's activation function). While having many desirable properties of associative memory, both of these classical systems suffer from a small memory storage capacity, which scales linearly with the number of input features.<ref name="Hopfield1982" /> In contrast, by increasing the number of parameters in the model so that there are not just pair-wise but also higher-order interactions between the neurons, one can increase the memory storage capacity.<ref name="Horn-Usher">{{Cite journal|last1=Horn|first1=D|last2=Usher|first2=M|date=1988|title=Capacities of multiconnected memory models|url=https://doi.org/10.1051/jphys:01988004903038900|journal=J. Phys. France|volume=49|issue=3|pages=389–395|doi=10.1051/jphys:01988004903038900}}</ref><ref name="Burns-Fukai">{{Cite journal|last1=Burns|first1=Thomas|last2=Fukai|first2=Tomoki|date=2023|title=Simplicial Hopfield networks|url=https://openreview.net/forum?id=_QLsH8gatwx|arxiv=2305.05179|journal=International Conference on Learning Representations|volume=11}}</ref>

Dense Associative Memories<ref name=":1">{{Cite journal|last1=Krotov|first1=Dmitry|last2=Hopfield|first2=John|date=2016|title=Dense Associative Memory for Pattern Recognition|arxiv=1606.01164|journal=Neural Information Processing Systems|volume=29|pages=1172–1180}}</ref> (also known as the modern Hopfield networks<ref name=":3">{{Cite journal|last1=Ramsauer|first1=Hubert|last2=Schäfl|first2=Bernhard|display-authors=1|date=2021|title=Hopfield Networks is All You Need|arxiv=2008.02217|journal=International Conference on Learning Representations}}</ref>) are generalizations of the classical Hopfield Networks that break the linear scaling relationship between the number of input features and the number of stored memories. This is achieved by introducing stronger non-linearities (either in the energy function or neurons' activation functions) leading to super-linear<ref name=":1" /> (even an exponential<ref name=":2">{{Cite journal|last1=Mete|first1=Demircigil|last2=Heusel|first2=Judith|display-authors=1|date=2017|title=On a model of associative memory with huge storage capacity.|url=https://link.springer.com/article/10.1007/s10955-017-1806-y|journal=Journal of Statistical Physics|volume=168|issue=2|pages=288–299|doi=10.1007/s10955-017-1806-y|arxiv=1702.01929|bibcode=2017JSP...168..288D |s2cid=119317128}}</ref>) memory storage capacity as a function of the number of feature neurons, in effect increasing the order of interactions between the neurons.<ref name="Horn-Usher" /><ref name="Burns-Fukai" /> The network still requires a sufficient number of hidden neurons.<ref name=":4">{{Cite journal|last1=Krotov|first1=Dmitry|last2=Hopfield|first2=John|date=2021|title=Large associative memory problem in neurobiology and machine learning|arxiv=2008.06996|journal=International Conference on Learning Representations}}</ref>

The key theoretical idea behind dense associative memory networks is to use an energy function and an update rule that is more sharply peaked around the stored memories in the space of neuron's configurations compared to the classical model,<ref name=":1" /> as demonstrated when the higher-order interactions and subsequent energy landscapes are explicitly modelled.<ref name="Burns-Fukai" />

=== Discrete variables ===
A simple example<ref name=":1" /> of the modern Hopfield network can be written in terms of [[binary data|binary variables]] <math>V_i</math> that represent the active <math>V_i=+1</math> and inactive <math>V_i=-1</math> state of the model neuron <math>i</math>.<math display="block" id="DAM_Energy">E = - \sum\limits_{\mu = 1}^{N_\text{mem}} F\Big(\sum\limits_{i=1}^{N_f}\xi_{\mu i} V_i\Big)</math>In this formula the weights <math display="inline">\xi_{\mu i}</math> represent the matrix of memory vectors (index <math>\mu = 1...N_\text{mem}</math> enumerates different memories, and index <math>i=1...N_f</math> enumerates the content of each memory corresponding to the <math>i</math>-th feature neuron), and the function <math>F(x)</math> is a rapidly growing non-linear function. The update rule for individual neurons (in the asynchronous case) can be written in the following form <math display="block" id="DAM_update_rule">V^{(t+1)}_i = Sign\bigg[ \sum\limits_{\mu=1}^{N_\text{mem}} \bigg(F\Big(\xi_{\mu i} + \sum\limits_{j\neq i}\xi_{\mu j} V^{(t)}_j\Big) - F\Big(-\xi_{\mu i} + \sum\limits_{j\neq i}\xi_{\mu j} V^{(t)}_j\Big) \bigg)\bigg]</math>which states that in order to calculate the updated state of the <math display="inline">i</math>-th neuron the network compares two energies: the energy of the network with the <math>i</math>-th neuron in the ON state and the energy of the network with the <math>i</math>-th neuron in the OFF state, given the states of the remaining neuron. The updated state of the <math>i</math>-th neuron selects the state that has the lowest of the two energies.<ref name=":1" />

In the limiting case when the non-linear energy function is quadratic <math>F(x) = x^2</math> these equations reduce to the familiar energy function and the update rule for the classical binary Hopfield Network.<ref name="Hopfield1982" />

The memory storage capacity of these networks can be calculated for random binary patterns. For the power energy function <math>F(x)=x^n</math> the maximal number of memories that can be stored and retrieved from this network without errors is given by<ref name=":1" /><math display="block">N^{max}_{\text{mem}}\approx \frac{1}{2 (2n-3)!!} \frac{N_f^{n-1}}{\ln(N_f)}</math>For an exponential energy function <math display="inline">F(x)=e^x</math> the memory storage capacity is exponential in the number of feature neurons<ref name=":2" /><math display="block">N^{max}_{\text{mem}}\approx 2^{N_f/2}</math>
[[File:Modern Hopfield Network.png|thumb|632x632px|Fig. 1: An example of a continuous modern Hopfield network with <math display="inline">N_f=5</math>  feature neurons and <math>N_\text{mem}=11</math> memory (hidden) neurons with symmetric synaptic connections between them.]]

=== Continuous variables ===
Modern Hopfield networks or dense associative memories can be best understood in continuous variables and continuous time.<ref name=":3" /><ref name=":4" /> Consider the network architecture, shown in Fig.1, and the equations for neuron's states evolution<ref name=":4" />{{NumBlk2|:|<math display="block" id="dynamical equations">\begin{cases}
\tau_f \frac{d x_i}{dt} = \sum\limits_{\mu=1}^{N_h} \xi_{i \mu} f_\mu - x_i + I_i\\
\tau_h \frac{d h_\mu}{dt} = \sum\limits_{i=1}^{N_f} \xi_{\mu i} g_i - h_\mu
\end{cases}</math>|1}}where the currents of the feature neurons are denoted by <math display="inline">x_i</math>, and the currents of the memory neurons are denoted by <math>h_\mu</math> (<math>h</math> stands for hidden neurons). There are no synaptic connections among the feature neurons or the memory neurons. A matrix <math>\xi_{\mu i}</math> denotes the strength of synapses from a feature neuron <math>i</math> to the memory neuron <math>\mu</math>. The synapses are assumed to be symmetric, so that the same value characterizes a different physical synapse from the memory neuron <math>\mu</math> to the feature neuron <math>i</math>. The outputs of the memory neurons and the feature neurons are denoted by <math>f_\mu</math> and <math>g_i</math>, which are non-linear functions of the corresponding currents. In general these outputs can depend on the currents of all the neurons in that layer so that <math>f_\mu = f(\{h_\mu\})</math> and <math display="inline">g_i = g(\{x_i\})</math>. It is convenient to define these activation functions as derivatives of the Lagrangian functions for the two groups of neurons{{NumBlk2|:|<math display="block" id="Lagrangian_def">f_\mu = \frac{\partial L_h}{\partial h_\mu},\ \ \ \  \text{and}\ \ \ \ g_i = \frac{\partial L_x}{\partial x_i}</math>|2}}This way the specific form of the equations for neuron's states is completely defined once the Lagrangian functions are specified. Finally, the time constants for the two groups of neurons are denoted by <math>\tau_f</math> and <math>\tau_h</math>, <math>I_i</math> is the input current to the network that can be driven by the presented data.  
[[File:Effective theory of Modern Hopfield Networks.png|thumb|827x827px|Fig. 2: Effective theory on the feature neurons for various common choices of the Lagrangian functions. Model A reduces to the models studied in<ref name=":1" /><ref name=":2" /> depending on the choice of the activation function, model B reduces to the model studied in,<ref name=":3" /> model C reduces to the model of.<ref name=":4" /> F is a "[[Smoothness|smooth]] enough" function.<ref name=":1" />]]
General systems of non-linear differential equations can have many complicated behaviors that can depend on the choice of the non-linearities and the initial conditions. For Hopfield Networks, however, this is not the case - the dynamical trajectories always converge to a fixed point attractor state. This property is achieved because these equations are specifically engineered so that they have an underlying energy function<ref name=":4" /> {{NumBlk2|:|<math display="block" id="energy">E(t) = \Big[\sum\limits_{i=1}^{N_f} (x_i-I_i) g_i - L_x \Big] + \Big[\sum\limits_{\mu=1}^{N_h}  h_\mu f_\mu - L_h \Big] - \sum\limits_{\mu, i} f_\mu \xi_{\mu i} g_i</math>|3}}The terms grouped into square brackets represent a Legendre transform of the Lagrangian function with respect to the states of the neurons. If the Hessian matrices of the Lagrangian functions are positive semi-definite, the energy function is guaranteed to decrease on the dynamical trajectory<ref name=":4" /> {{NumBlk2|:|<math display="block" id="energy_decrease">\frac{dE(t)}{dt}= - \tau_f \sum\limits_{i,j=1}^{N_f} \frac{d x_i}{dt} \frac{\partial^2 L_x}{\partial x_i \partial x_j} \frac{d x_j}{dt} - \tau_h \sum\limits_{\mu,\nu = 1}^{N_h}  \frac{d h_\mu}{dt} \frac{\partial^2 L_h}{\partial h_\mu \partial h_\nu} \frac{d h_\nu}{dt}  \leq 0</math>|4}} This property makes it possible to prove that the system of dynamical equations describing temporal evolution of neurons' activities will eventually reach a fixed point attractor state.

In certain situations one can assume that the dynamics of hidden neurons equilibrates at a much faster time scale compared to the feature neurons, <math display="inline">\tau_h\ll\tau_f</math>. In this case the steady state solution of the second equation in the system ({{EquationNote|1}}) can be used to express the currents of the hidden units through the outputs of the feature neurons. This makes it possible to reduce the general theory ({{EquationNote|1}}) to an effective theory for feature neurons only. The resulting effective update rules and the energies for various common choices of the Lagrangian functions are shown in Fig.2. In the case of log-sum-exponential Lagrangian function the update rule (if applied once) for the states of the feature neurons is the attention mechanism<ref name=":3" /> commonly used in many modern AI systems (see Ref.<ref name=":4" /> for the derivation of this result from the continuous time formulation).

=== Relationship to classical Hopfield network with continuous variables ===
Classical formulation of continuous Hopfield Networks<ref name=":0" /> can be understood<ref name=":4" /> as a special limiting case of the modern Hopfield networks with one hidden layer. Continuous Hopfield Networks for neurons with graded response are typically described<ref name=":0" /> by the dynamical equations {{NumBlk2|:|<math display="block">\tau_f \frac{d x_i}{dt} = \sum\limits_{j=1}^{N_f}T_{ij} V_j - x_i + I_i</math>|5}} and the energy function {{NumBlk2|:|<math display="block">E = -\frac{1}{2}\sum\limits_{i,j=1}^{N_f} T_{ij} V_i V_j - \sum\limits_{i=1}^{N_f} V_i I_i + \sum\limits_{i=1}^{N_f} \int\limits^{V_i} g^{-1}(z)\,dz</math>|6}} where <math display="inline">V_i = g(x_i)</math>, and <math>g^{-1}(z)</math> is the inverse of the activation function <math>g(x)</math>. This model is a special limit of the class of models that is called models A,<ref name=":4" /> with the following choice of the Lagrangian functions {{NumBlk2|:|<math display="block">L_v = \sum\limits_{i=1}^{N_f}\int\limits^{x_i} g(x) dx,\ \ \ \ \ \text{and}\ \ \ \ \  L_h = \frac{1}{2} \sum\limits_{\mu=1}^{N_h} h_\mu^2</math>|7}} that, according to the definition ({{EquationNote|2}}), leads to the activation functions {{NumBlk2|:|<math display="block">V_i = g(x_i), \ \ \ \ \ \text{and}\ \ \ \ \ f_\mu = h_\mu</math>|8}} If we integrate out the hidden neurons the system of equations ({{EquationNote|1}}) reduces to the equations on the feature neurons ({{EquationNote|5}}) with <math>T_{ij} = \sum\limits_{\mu=1}^{N_h} \xi_{\mu i }\xi_{\mu j}</math>, and the general expression for the energy ({{EquationNote|3}}) reduces to the effective energy {{NumBlk2|:|<math display="block">E = -\frac{1}{2} \sum\limits_{i,j=1}^{N_f} T_{ij} V_i V_j - \sum\limits_{i=1}^{N_f} V_i I_i +\sum\limits_{i=1}^{N_f} \Big( x_i V_i - \int\limits^{x_i} g(x)dx \Big)</math>|9}} While the first two terms in equation ({{EquationNote|6}}) are the same as those in equation ({{EquationNote|9}}), the third terms look superficially different. In equation ({{EquationNote|9}}) it is a Legendre transform of the Lagrangian for the feature neurons, while in ({{EquationNote|6}}) the third term is an integral of the inverse activation function. Nevertheless, these two expressions are in fact equivalent, since the derivatives of a function and its Legendre transform are inverse functions of each other. The easiest way to see that these two terms are equal explicitly is to differentiate each one with respect to <math>x_i</math>. The results of these differentiations for both expressions are equal to <math>x_i g(x_i)'</math>. Thus, the two expressions are equal up to an additive constant. This completes the proof<ref name=":4" /> that the classical Hopfield Network with continuous states<ref name=":0" /> is a special limiting case of the modern Hopfield network ({{EquationNote|1}}) with energy ({{EquationNote|3}}).

=== General formulation of the modern Hopfield network ===
[[File:HAM Full Connect.png|thumb|390x390px|Fig. 3: The connectivity diagram of the fully-connected modern Hopfield network consisting of five neurons. The synaptic weights are described by a symmetric matrix <math>W_{IJ}</math>.]]
Biological neural networks have a large degree of heterogeneity in terms of different cell types. This section describes a mathematical model of a fully connected modern Hopfield network assuming the extreme degree of heterogeneity: every single neuron is different.<ref name=":5" /> Specifically, an energy function and the corresponding dynamical equations are described assuming that each neuron has its own activation function and kinetic time scale.  The network is assumed to be fully connected, so that every neuron is connected to every other neuron using a symmetric matrix of weights <math>W_{IJ}</math>, indices <math>I</math> and <math>J</math> enumerate different neurons in the network, see Fig.3. The easiest way to mathematically formulate this problem is to define the architecture through a Lagrangian function <math>L(\{x_I\})</math> that depends on the activities of all the neurons in the network. The activation function for each neuron is defined as a partial derivative of the Lagrangian  with respect to that neuron's activity {{NumBlk2|:|<math display="block">g_I = \frac{\partial L}{\partial x_I}</math>|10}} From the biological perspective one can think about <math>g_I</math> as an axonal output of the neuron <math>I</math>. In the simplest case, when the Lagrangian is additive for different neurons, this definition results in the activation that is a non-linear function of that neuron's activity. For non-additive Lagrangians this activation function can depend on the activities of a group of neurons. For instance, it can contain contrastive (softmax) or divisive normalization. The dynamical equations describing temporal evolution of a given neuron are given by<ref name=":5" /> {{NumBlk2|:|<math display="block">\tau_I \frac{dx_I}{dt} = \sum\limits_{J=1}^N W_{IJ} g_J - x_I</math>|11}} This equation belongs to the class of models called firing rate models in neuroscience. Each neuron <math>I</math> collects the axonal outputs <math>g_J</math> from all the neurons, weights them with the synaptic coefficients <math>W_{IJ}</math> and produces its own time-dependent activity <math>x_I</math>. The temporal evolution has a time constant <math>\tau_I</math>, which in general can be different for every neuron. This network has a global energy function<ref name=":5" /> {{NumBlk2|:|<math display="block">E = \sum\limits_{I=1}^N x_I g_I - L - \frac{1}{2} \sum\limits_{I,J=1}^N g_I W_{IJ} g_J </math>|12}} where the first two terms represent the Legendre transform of the Lagrangian function with respect to the neurons' currents <math>x_I</math>. The temporal derivative of this energy function can be computed on the dynamical trajectories leading to (see <ref name=":5">{{cite arXiv|last=Krotov|first=Dmitry|date=2021|title=Hierarchical Associative Memory|class=cs.NE|eprint=2107.06446}}</ref>  for details) {{NumBlk2|:|<math display="block">\frac{dE}{dt} = - \sum\limits_{I,K=1}^N \frac{dx_I}{dt} M_{IK} \frac{dx_K}{dt}\leq 0, \ \ \ \ \text{where}\ \ \ \ M_{IK} = \tau_I  \frac{\partial^2 L }{\partial x_I \partial x_K}</math>|13}} The last inequality sign holds provided that the matrix <math>M_{IK}</math> (or its symmetric part) is positive semi-definite. If, in addition to this, the energy function is bounded from below the non-linear dynamical equations are guaranteed to converge to a fixed point attractor state. The advantage of formulating this network in terms of the Lagrangian functions is that it makes it possible to easily experiment with different choices of the activation functions and different architectural arrangements of neurons. For all those flexible choices the conditions of convergence are determined by the properties of the matrix <math>M_{IJ}</math> and the existence of the lower bound on the energy function.
[[File:Hierarchical Associative Memory.png|thumb|519x519px|Fig. 4: The connectivity diagram of the layered Hierarchical Associative Memory network.<ref name=":5" /> Each layer can have different number of neurons, different activation function, and different time scales. The feedforward weights and feedback weights are equal.]]

=== Hierarchical associative memory network ===
The neurons can be organized in layers so that every neuron in a given layer has the same activation function and the same dynamic time scale. If we assume that there are no horizontal connections between the neurons within the layer (lateral connections) and there are no skip-layer connections, the general fully connected network ({{EquationNote|11}}), ({{EquationNote|12}}) reduces to the architecture shown in Fig.4. It has <math>N_\text{layer}</math> layers of recurrently connected neurons with the states described by continuous variables <math>x_i^{A}</math> and the activation functions <math>g_i^{A}</math>, index <math>A</math> enumerates the layers of the network, and index <math>i</math> enumerates individual neurons in that layer. The activation functions can depend on the activities of all the neurons in the layer. Every layer can have a different number of neurons <math>N_A</math>. These neurons are recurrently connected with the neurons in the preceding and the subsequent layers. The matrices of weights that connect neurons in layers <math>A</math> and <math>B</math> are denoted by <math>\xi^{(A,B)}_{ij}</math> (the order of the upper indices for weights is the same as the order of the lower indices, in the example above this means that the index <math>i</math> enumerates neurons in the layer <math>A</math>, and index <math>j</math> enumerates neurons in the layer <math>B</math>). The feedforward weights and the feedback weights are equal. The dynamical equations for the neurons' states can be written as<ref name=":5" /> {{NumBlk2|:|<math display="block">\tau_A \frac{dx_i^A}{dt} = \sum\limits_{j=1}^{N_{A-1}} \xi^{(A, A-1)}_{ij} g_j^{A-1} + \sum\limits_{j=1}^{N_{A+1}} \xi^{(A, A+1)}_{ij} g_j^{A+1} - x_i^A</math>|14}} with boundary conditions {{NumBlk2|:|<math display="block">g_i^0 =0, \ \ \ \ \ \text{and}\ \ \ \ \ g_i^{N_\text{layer}+1}=0</math>|15}} The main difference between these equations and those from the conventional feedforward networks is the presence of the second term, which is responsible for the feedback from higher layers. These top-down signals help neurons in lower layers to decide on their response to the presented stimuli. Following the general recipe it is convenient to introduce a Lagrangian function <math>L^A(\{x^A_i\})</math> for the <math>A</math>-th hidden layer, which depends on the activities of all the neurons in that layer.<ref name=":5" /> The activation functions in that layer can be defined as partial derivatives of the Lagrangian {{NumBlk2|:|<math display="block">g_i^A = \frac{\partial L^A}{\partial x_i^A}</math>|16}} With these definitions the energy (Lyapunov) function is given by<ref name=":5" /> {{NumBlk2|:|<math display="block">E = \sum\limits_{A=1}^{N_\text{layer}} \Big[ \sum\limits_{i=1}^{N_A} x_i^A g_i^A - L^{A}\Big] - \sum\limits_{A=1}^{N_\text{layer}-1} \sum\limits_{i=1}^{N_{A+1}} \sum\limits_{j=1}^{N_A} g_i^{A+1} \xi^{(A+1,A)}_{ij} g_j^A</math>|17}} If the Lagrangian functions, or equivalently the activation functions, are chosen in such a way that the Hessians for each layer are positive semi-definite and the overall energy is bounded from below, this system is guaranteed to converge to a fixed point attractor state. The temporal derivative of this energy function is given by<ref name=":5" /> {{NumBlk2|:|<math display="block">\frac{dE}{dt} = -\sum\limits_{A=1}^{N_\text{layer}} \tau_A \sum\limits_{i,j=1}^{N_A} \frac{dx_j^A}{dt} \frac{\partial^2 L^{A}}{\partial x_j^{A} \partial x_i^{A}} \frac{dx_i^A}{dt} \leq 0</math>|18}} Thus, the hierarchical layered network is indeed an attractor network with the global energy function. This network is described by a hierarchical set of synaptic weights that can be learned for each specific problem.

==See also==
* [[Associative memory (disambiguation)]]
* [[Autoassociative memory]]
* [[Boltzmann machine]] – like a Hopfield net but uses annealed Gibbs sampling instead of gradient descent
* [[Cognitive model#Associative memory|Dynamical systems model of cognition]]
* [[Ising model]]
* [[Hebbian theory]]

==References==
{{Reflist}}

{{Refbegin}}
* {{cite book |first=D.O. |last=Hebb |title=The Organization of Behavior: A Neuropsychological Theory |url=https://books.google.com/books?id=ddB4AgAAQBAJ |date=2005 |publisher=Psychology Press |isbn=978-1-135-63190-1 |orig-year=1949 |ref={{harvid|Hebb|1949}}}}
* {{cite book |first=John A. |last=Hertz |title=Introduction To The Theory Of Neural Computation |url=https://books.google.com/books?id=aEIPEAAAQBAJ&pg=PP5 |date=2018 |publisher=CRC Press |isbn=978-0-429-96821-1 |orig-year=1991 |ref={{harvid|Hertz|1991}}}}
* {{cite journal | last1 = McCulloch | first1 = W.S. | last2 = Pitts | first2 = W.H. | year = 1943 | title = A logical calculus of the ideas immanent in nervous activity | journal = Bulletin of Mathematical Biophysics | volume = 5 | issue = 4| pages = 115–133 | doi = 10.1007/BF02478259 }}
* {{cite journal | last1 = Polyn | first1 = S.M. | last2 = Kahana | first2 = M.J. | year = 2008 | title = Memory search and the neural representation of context | journal = Trends in Cognitive Sciences | volume = 12 | issue = 1| pages = 24–30 | doi=10.1016/j.tics.2007.10.010| pmid = 18069046 | pmc = 2839453 }}
* {{cite journal | last1 = Rizzuto | first1 = D.S. | last2 = Kahana | first2 = M.J. | s2cid = 7675117 | year = 2001 | title = An autoassociative neural network model of paired-associate learning | journal = Neural Computation | volume = 13 | issue = 9| pages = 2075–2092 | doi=10.1162/089976601750399317| pmid = 11516358 | citeseerx = 10.1.1.45.7929 }}
* {{cite book |first1=Rudolf |last1=Kruse |first2=Christian |last2=Borgelt |first3=Frank |last3=Klawonn |first4=Christian |last4=Moewes |first5=Matthias |last5=Steinbrecher |first6=Pascal |last6=Held |title=Computational Intelligence: A Methodological Introduction |url=https://books.google.com/books?id=yQVGAAAAQBAJ |date=2013 |publisher=Springer |isbn=978-1-4471-5013-8}}

{{Refend}}

== External links ==
{{commons|Hopfield net}}
* {{cite book |chapter-url=http://page.mi.fu-berlin.de/rojas/neural/chapter/K13.pdf |chapter=13. The Hopfield model |url=http://page.mi.fu-berlin.de/rojas/neural/index.html.html |title=Neural Networks – A Systematic Introduction |first=Raul |last=Rojas |date=12 July 1996 |publisher=Springer |isbn=978-3-540-60505-8}}
*[http://www.heatonresearch.com/aifh/vol3/hopfield.html Hopfield Network Javascript]
*[http://to-campos.planetaclix.pt/neural/hope.html The Travelling Salesman Problem] {{Webarchive|url=https://web.archive.org/web/20150530022419/http://to-campos.planetaclix.pt/neural/hope.html |date=2015-05-30 }} – Hopfield Neural Network JAVA Applet
*{{cite journal |journal=Scholarpedia |title=Hopfield network |first=John |last=Hopfield|year=2007 |volume=2 |issue=5 |page=1977 |doi=10.4249/scholarpedia.1977 |bibcode=2007SchpJ...2.1977H |doi-access=free }}
* {{cite web |date=November 7, 2020 |title=Don't Forget About Associative Memories |url=https://thegradient.pub/dont-forget-about-associative-memories/ |access-date=September 27, 2024 |website=The Gradient |ref={{sfnref|The Gradient|2020}}}}
*{{cite web |url=http://www.tristanfletcher.co.uk/DLVHopfield.pdf |title=Hopfield Network Learning Using Deterministic Latent Variables |type=Tutorial |first=Tristan |last=Fletcher|archive-url=https://web.archive.org/web/20111005202201/http://www.tristanfletcher.co.uk/DLVHopfield.pdf |archive-date=2011-10-05 }}

{{Differentiable computing}}
{{Stochastic processes}}

[[Category:Neural network architectures]]