Pattern retrieval and energies in a Hopfield network
A Hopfield network is a type of neural network that can be used to store patterns. This Hopfield network is implemented as follows:
- Each cell in the grids below corresponds to a node. Each cell is either grey ($c_i = 1$) or white ($c_i = -1$).
- The weight matrix is defined as $ w_{ij} = \frac{1}{N} \sum_{k=1}^{N} x^{(k)}_{i} x^{(k)}_j$ where $x$ is a stored pattern and $N$ is the total number of stored pattern. It is symmetric ($w_{ij} = w_{ji}$) and there are no self connections ($w_{ii} = 0$).
- The network is synchronously updated, with the result of one update being presented as the output.
- The energy of the network is defined by $E = -\frac{1}{2} \sum_{ij} w_{ij} c_{i} c_{j}$ where $c$ is the current displayed pattern of the network.
Observations
- Storing the same pattern more than once will lower that pattern's energy.
- Too many similar patterns being stored may result in some superposition of the patterns being retrieved (a 'mixture state').
- Unstable states that switch between each other can be observed.
- The 'inverse' pattern has the same energy as the real pattern.
Hopfield network simulation