Hey, I am working on a program that uses the perceptron algorithm to do pattern recognition and classification. Simply put I use a 2d grid in which each pixel is either on (1) or off (-1). using the following rules and algorithm taken from some reference.: w(x) = w(x) + LR * R * v(x) b = b + LR * R w - weigth vector v - value vector LR learning rate r expected result The pseudo code for the algorithm:

while (1 ) {

   updates_made = 0

   For each Example in Training Set {

     /* o is b + SUM( v(x) * w(x) for all x ) */
     o = awnser ( Example, w, b )

     if ( o != R ) {
         for x = 0 to v.count {
             w(x) = w(x) + LR * R * v(x)
             b = b + LR * R
         }
         updates_made = 1
     }
   }

   if (updates_made==0) {
      break
   }

Problem: train it to recognize a pattern. Then ask if a pattern where one of the active pixels is unactive comparting to the original pattern, it says its the same. The problem is it keeps saying its the same until all the the pixels of the original have been deleted. The sum is giving very high results and taking out one of the active pixels takes a very small value from the ottal sum. Any ideas? Thanks.

Note that the awnser function returns

sum = b + SUM ( v(x) * w (x ) for all x )

if sum > 0
return 1
else
return0

Thanks.

It is behaving as it should. Your expectations do not match what it is doing. It is successfully recognizing all patterns it was trained to recognize. It is also recognizing all patterns, including those that it was not trained to recognize. You never trained it to not recognize those patterns.

It is important to have examples that do not match the pattern. Classification is about picking which category an input belongs in. Your examples only show things that have some property P. It is also important to show things that do not have property P to allow the classifier to know what should be put in category "not P". An artificial neural network designed to recognize numbers might have one output for each number, and the training set would train it to output a 1 for the output corresponding to the input number and a 0 for all the other outputs. Then after training you can give it a new input and see which output is the greatest to decide what the input was. If several outputs are high, then the input may have been ambiguous.

In your case, you can try training the network to recognize things that do not match your pattern by outputting -1, just like you're training it to recognize things that do match your pattern by outputting 1.

You should also realize that perceptrons are only capable of learning linearly separable functions. The classic example is that a perceptron can't learn xor, and even a network of linear perceptrons like yours, in any topology, cannot learn xor. If the function you want it to learn is not linearly separable then you'll need to move on to something more advanced, like multilayer perceptrons with a non-linear activation function.

Quote: Original post by Vorpy
If the function you want it to learn is not linearly separable then you'll need to move on to something more advanced, like multilayer perceptrons with a non-linear activation function.

Or something that is actually good at classification problems *cough*, like SVM's. If I may add something to what Vorpy said, when testing a classification machine, it is important to test it on both the training set *and* an external testing set, containing only items the machine has not been trained with. Otherwise you'll get badddddd surprises later!

Hey, thanks for your comments.

Quote:
It is important to have examples that do not match the pattern. Classification is about picking which category an input belongs in. Your examples only show things that have some property P. It is also important to show things that do not have property P to allow the classifier to know what should be put in category "not P".

Quote:
In your case, you can try training the network to recognize things that do not match your pattern by outputting -1, just like you're training it to recognize things that do match your pattern by outputting 1.

Thanks, Will try this.

Is it better not to use random when generating the noisy patterns?

Example of how its generating:
5% random changes (for YES)
95% random changes (for NO)

Could use the other characters as NO examples, however I want to have this working for a single character first.

Thanks for your comments.

Ouch found a crazy bug

for x = 0 to v.count {             w(x) = w(x) + LR * R * v(x)             b = b + LR * R}

this is adding to the bias v.count times, hence the bias was giving crazy values.