Hopfield Networks as Associative Memory

(Documentation of the programs xhfn and whfn)

Contents

about.png

Introduction

Hopfield networks are a special kind of recurrent neural networks that can be used as associative memory. Associative memory is memory that is addressed through its contents. That is, if a pattern is presented to an associative memory, it returns whether this pattern coincides with a stored pattern. The coincidence need not be perfect, though. An associative memory may also return a stored pattern that is similar to the presented one, so that noisy input can also be recognized.

Hopfield networks are used as associative memory by exploiting the property that they possess stable states, one of which is reached by carrying out the normal computations of a Hopfield network. If the connection weights of the network are determined in such a way that the patterns to be stored become the stable states of the network, a Hopfield network produces for any input pattern a similar stored pattern. Thus noisy patterns can be corrected or distorted patterns can still be recognized.

Details about the structure of a Hopfield network and about how to construct a Hopfield network that stores given patterns can be found in most textbooks on artificial neural networks.

back to the top top

Pattern Storage and Retrieval

With the programs xhfn and whfn the storage and retrieval of simple two-dimensional bit patterns in a Hopfield networks can be demonstrated. The main window of the program displays a grid of neurons. The fields of this grid are colored according to the activation of the corresponding neuron: grey corresponds to an activation of 1, white to an activation of -1. The activation of a neuron may be changed by clicking on the corresponding field. In this way a pattern can be entered. The pattern may then be stored by selecting the menu entry "Actions > Store Pattern" (or, in the Unix version, by pressing "t"). What patterns have already been stored can be reviewed by repeatedly selecting "Actions > Next Pattern" (or, in the Unix version, by pressing "n"). Already stored may also be deleted again by selecting "Actions > Delete Pattern" (or, in the Unix version, by pressing "d"), when the pattern to be deleted is shown in the main window.

Once all desired patterns have been stored, the retrieval capacities of the network may be tested. To do so set the neuron activations to a slightly distorted version of the pattern and start the update process by selecting "Actions > Start Update" (or, in the Unix version, by pressing "x"). The activations of the neurons may also be initialized randomly by selecting "Actions > Initialize" (or, in the Unix version, by pressing "i"). The update process then brings the network into a stable state, which (hopefully) is the stored pattern that is similar to the one entered. It may, however, also converge to a spurious stable state that is not similar to any stored pattern.

As an example, let us consider the set of eight characters shown below. These patterns can be found in the file hfnd/ex/numbers.hfn, which may be loaded through "File > Load Network".

pat0.png pat1.png
pat2.png pat3.png
pat4.png pat6.png
pato.png pat9.png

In order to test the Hopfield network that stores these patterns, select "Actions > Initialize", which sets the neuron activations randomly. Then start the computations of the network with "Actions > Start Update" and follow how a stored pattern is reconstructed. However, the reconstruction is not always perfect. Due to the method used to store the patterns, the complement of the stored patterns are also stable states, so that sometimes such a complement is reached. In addition there are some spurious stable states that do not correspond (exactly) to stored patterns.

back to the top top

Parameters

Dialog to set the neuron grid parameters

If the menu entry Settings > Grid... is selected (or, in the Unix version, if `g' for `grid' is pressed), a dialog box is opened in which the grid parametersmay be entered (this dialog box is shown below).

grid.png

In the first two input fields the width and the height of the Hopfield network may be entered. The third field allows a specification of the size of the square that is used to visualize the activation of a neuron.

Dialog to set the update parameters

If the menu entry Settings > Parameters... is selected (or, in the Unix version, if `p' for `parameters' is pressed), a dialog box is opened in which the update parameters of the Hopfield network can be specified (this dialog box is shown below).

params.png

The update mode is either "single neuron", which means that the Hopfield network is redisplayed after each update of a neuron, or "all neurons", which means that the Hopfield network is redisplayed only after all neurons have been updated. The delay value controls the amount of time that has to pass between to redisplays of the Hopfield network, so that the computations can be followed conveniently.

Loading and saving a Hopfield network

A Hopfield network (or rather the set of stored patterns) can be saved to a file and reloaded later. To do so one has to select the menu entries File > Save Network... or File > Load Network..., respectively. (Alternatively, in the Unix version, one may press `s' for `save' and `l' for `load'.) The file selector box that is opened, if one of these menu entries are selected, is shown below. (In the Windows version the standard Windows file selector box is used.)

save.png

The edit field below 'Directory/Filter' shows the current path, the list box below 'Files' the files in the current directory matching the current file name pattern. (A file name pattern, which may contain the usual wildcards '*' and '?', may be added to the path. When the file name pattern has been changed, pressing the 'Update' button updates the file list.) The line below 'Selection' shows the currently selected file. It may be chosen from the file list or typed in.

back to the top top

License

(MIT license, or more precisely Expat License; to be found in the file mit-license.txt in the directory hfnd/doc in the source package, see also opensource.org and wikipedia.org)

© 2002-2016 Christian Borgelt

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

back to the top top

Download

Download pagewith most recent version.

back to the top top

Contact

E-mail: christian@borgelt.net
Website: www.borgelt.net
back to the top top

© 2002-2016 Christian Borgelt