View source for RAMnets

{{no footnotes|date=June 2018}}
'''RAMnets''' is one of the oldest practical neurally inspired classification [[algorithms]]. The RAMnets  is also known as a type of "''n''-tuple recognition method" or "weightless neural network".

==Algorithm==
[http://www.haralick.org/ML/NCRG_95_013.pdf Consider (let us say ''N'') sets of n distinct bit locations are selected randomly.] These are the ''n''-tuples. The restriction of a pattern to an n-tuple can be regarded as an ''n''-bit number which, together with the identity of the ''n''-tuple, constitutes a `feature' of the pattern. The standard ''n''-tuple recognizer operates simply as follows:

''A pattern is classified as belonging to the class for which it has the most features in common with at least one training pattern of that class.''

This is the <math>\Theta</math>= 0 case of a more general rule whereby the class assigned to unclassified pattern u is

   <math>\begin{align} \underset{c}argmax(\sum_{i=1}^N\Theta(\sum_{v\in D_{c}}\delta(\alpha_{i}(u),\alpha_{i}(v))))\end{align}</math>

where D<sub>c</sub> is the set of training patterns in class c, <math>\Theta(x)</math>= x for <math>0\leq x\leq \theta</math> ,<math>\Theta(x)=\theta</math> for <math>x\geq\theta</math>,<math>\delta_{i,j}</math> is the [[Kronecker delta]](<math>\delta_{i,j}</math>=1 if i=j and 0 otherwise.)and <math>(\alpha_{i}(u))</math>is the i<sup>th</sup> feature of the pattern u:

   <math>\sum_{j=0}^{n-1}u_\eta i(j)2^{j}</math>

Here u<sub>k</sub> is the k<sup>th</sup> bit of u and <math>u_\eta i (j)</math>is the j<sup>th</sup> bit location of the i<sup>th</sup> n-tuple.

With C classes to distinguish, the system can be implemented as a network of NC nodes, each of which is a random access memory (RAM); hence the term ''RAMnet.'' The memory content <math>m_{ci\alpha}</math> at address <math>\alpha</math>  of the i<sup>th</sup> node allocated to class c is set to

    <math>m_{ci\alpha}</math> = <math>\Theta(\sum_{v\in D_{c}}\delta(\alpha,\alpha_{i}(v)))</math>

In the usual <math>\theta</math> = 1 case, the 1-bit content of <math>m_{ci\alpha}</math> is set if any pattern of D<sub>c</sub> has feature <math>\alpha </math> and unset otherwise. Recognition is  accomplished by summing the contents of the nodes of each class at the addresses given by the features of the unclassified pattern. That is, pattern '''u''' is assigned to class

    <math>\begin{align} \underset{c}argmax(\sum_{i=1}^N m_{ci\alpha}(u)) \end{align}</math>

==RAM-discriminators and WiSARD ==
The RAMnets formed the basis of a commercial product known as WiSARD (Wilkie, Stonham and Aleksander Recognition Device) was the first artificial neural network machine to be patented.

A RAM-discriminator consists of a set of {{var|X}} one-bit word RAMs with {{var|n}} inputs and a summing device (Σ). Any such RAM-discriminator can receive a binary pattern of X⋅n bits as input. The RAM input lines are connected to the input pattern by means of a biunivocal pseudo-random mapping. The summing device enables this network of RAMs to exhibit – just like other ANN models based on synaptic weights – generalization and noise tolerance.

In order to train the discriminator one has to set all RAM memory locations to 0 and choose a training set formed by binary patterns of X⋅n bits. For each training pattern, a 1 is stored in the memory location of each RAM addressed by this input pattern. Once the training of patterns is completed, RAM memory contents will be set to a certain number of 0’s and 1’s.

The information stored by the RAM during the training phase is used to deal with previous unseen patterns. When one of these is given as input, the RAM memory contents addressed by the input pattern are read and summed by Σ. The number {{var|r}} thus obtained, which is called the discriminator response, is equal to the number of RAMs that output 1. r reaches the maximum {{var|X}} if the input belongs to the training set. {{var|r}} is equal to 0 if no ''n''-bit component of the input pattern appears in the training set (not a single RAM outputs 1). Intermediate values of r express a kind of “similarity measure” of the input pattern with respect to the patterns in the training set.

A system formed by various RAM-discriminators is called WiSARD. Each RAM-discriminator is trained on a particular class of patterns, and classification by the multi-discriminator system is performed in the following way. When a pattern is given as input, each RAM-discriminator gives a response to that input. The various responses are evaluated by an algorithm which compares them and computes the relative confidence {{var|c}} of the highest response (e.g., the difference d between the highest response and the second highest response, divided by the highest response). A schematic representation of a RAM-discriminator and a 10 RAM-discriminator WiSARD is shown in Figure 1.<ref>{{Cite book|title=Advances in computational intelligence and learning : 17th European Symposium on Artificial Neural Networks; ESANN 2009; Bruges, Belgium, April 22-23-24, 2009; proceedings|date=2009|publisher=d-side|others=Verleysen, Michel, Université catholique de Louvain, ESANN (17 2009.04.22-24 Bruges), European Symposium on Artificial Neural Networks (17 2009.04.22-24 Bruges)|isbn=978-2930307091|location=Evere|oclc=553956424}}</ref>

== See also ==
* [[Artificial Neural Network]]
* [[Kronecker delta]]
* [[Pattern Recognition]]
* [[Unsupervised learning]]
* [[Erlang distribution]]
* [[Machine learning]]
* [[Erlang (unit)]]

== References ==
{{reflist}}
* Michal Morciniec and Richard Rohwer(1995) "[http://www.haralick.org/ML/NCRG_95_013.pdf The n-tuple Classifier: Too Good to Ignore]"
* {{cite book |first1=Trevor |last1=Hastie |first2=Robert |last2=Tibshirani |title=The Elements of Statistical Learning: Data mining, Inference, and Prediction |year=2009 |publisher=Springer| location=New York |isbn=978-0-387-84857-0 |pages=485–586 |doi=10.1007/978-0-387-84858-7_14}}
* {{cite book |editor1=[[Geoffrey Hinton|Hinton, Geoffrey]] |editor2=[[Terrence J. Sejnowski|Sejnowski, Terrence J.]] |year=1999 |title=Unsupervised Learning: Foundations of Neural Computation |publisher=[[MIT Press]] |isbn=0-262-58168-X}} (This book focuses on unsupervised learning  in [[neural network]]s)
*{{cite journal|title=Generalization in probabilistic RAM nets|first=.G. Clarkson|last1= Y. Guan, J.G. Taylor, D. Gorse|journal=IEEE Transactions on Neural Networks|volume=4|issue=2|year=1993|doi=10.1109/72.207603|pmid=18267737|pages=360–363}}
*[https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2009-6.pdf A brief introduction to Weightless NeuralSystems] (2009)  

==Further reading==
# {{Cite book|title=N-Tuple Neural Networks|last=N. M. Allinson|first=A. R. Kolcz|publisher=Springer, Boston, MA|year=1997|isbn=978-1-4615-6099-9|location=Springer Science+Business Media New York}}
# {{cite book|last=Fukunaga|first=Keinosuke|title=Introduction to Statistical Pattern Recognition|edition=2nd|year=1990|publisher=Academic Press|location=Boston|isbn=0-12-269851-7|url-access=registration|url=https://archive.org/details/introductiontost1990fuku}}
# {{cite book|last1=Hornegger|first1=Joachim|last2=Paulus|first2=Dietrich W. R.|title=Applied Pattern Recognition: A Practical Introduction to Image and Speech Processing in C++|edition=2nd|year=1999|publisher=Morgan Kaufmann Publishers|location=San Francisco|isbn=3-528-15558-2}}
# [http://www.egmont-petersen.nl/classifiers.htm An introductory tutorial to classifiers (introducing the basic terms, with numeric example)]

[[Category:Unsupervised learning]]