ANN - learning AND / OR - Artificial Intelligence

Author

122

August 25, 2005 07:49 PM

Hi all, my first time here so start with: I apologize for my bad english but i'm not a native english speaker... i'll try to do my best.. :D I wrote a little AAN with this characteristics: - It has to learn bit to bit AND and OR, so it takes two input (x1,x2) and calculate d = x1*w1 + x2*w2; (w1,w2) -> weights.. - the treshold i choosed is 0 and it is compared with d = x1 * w1 + x2 * w2, then if(d <= 0) d = 0; else d = 1; - weights are modified by this formulas wi(t+1) = wi(t) + n * (p - d) * xi with: n = learning rate, p = expected value, d = aim value. . - the program takes in inupt a file (exercise book) with this syntax (eg): 1 0 0 0 0 1 ecc and it means (first line): 1: first operand; 0: second operand; 0: expected result. And so on. All this works fine if this simple feed-forward perceptron try to learn how manage OR (learning rate 0.2… less than 20 example are enough), but it doesn't work if i try to give it an AND exercise book… The source is something like that (again apology: ugly coding style, don't try this at home! :D) <!–STARTSCRIPT–><!–source lang="cpp"–><div class="source"><pre> //main.cpp #include "main.h" #include "training.h" struct exercise ex[ESEMPI]; struct weight w; FILE *book = NULL; // exercise book int main(int argc, char *argv[]) { int n = 0; double ris; char filename[20]; w.wa = wrand(); w.wb = wrand(); cout << "Init weights: " << w.wa << " " << w.wb << endl; cout << "Book filename: "; cin >> filename; book = fopen(filename, "r"); if(book == NULL) { cout << "fopen error" << endl; exit(-1); } for(int i = 0; i < ESEMPI; i++) n = getexercise(&ex, &w, n); fclose(book); cout << "Final weight: " << w.wa << " " << w.wb << endl; for(;;) { cout << endl << "First operand: "; cin >> ex[0].a; cout << endl << "Second operand: "; cin >> ex[0].b; ris = (ex[0].a * w.wa) + (ex[0].b * w.wb); if(ris < 0) ris = 0; if(ris > 0) ris = 1; cout << endl << ex[0].a << " OP " << ex[0].b << " = " << ris << endl; } return 0; } </pre></div><!–ENDSCRIPT–> <!–STARTSCRIPT–><!–source lang="cpp"–><div class="source"><pre> //main.h #include <stdlib.h> #include <stdio.h> #include <iostream> #define ESEMPI 20 // book size using namespace std; </pre></div><!–ENDSCRIPT–> <!–STARTSCRIPT–><!–source lang="cpp"–><div class="source"><pre> //training.cpp #include "training.h" double wrand() // return a random number to initialize the weights { return (rand() / ((double)RAND_MAX + 1)); } int getexercise(struct exercise *ex, struct weight *w, int n) { // open an exercise book and fills an array of // struct exercise extern FILE *book; ex->a = (fgetc(book) - 48); fseek(book, n+=2, SEEK_SET); ex->b = (fgetc(book) - 48); fseek(book, n+=2, SEEK_SET); ex->r = (fgetc(book) - 48); fseek(book, n+=3, SEEK_SET); learning(ex, w); return n; } void learning(struct exercise *ex, struct weight *w) { double d; double deltaa; double deltab; const double n = 0.2; d = (ex->a * w->wa) + (ex->b * w->wb); if(d <= 0) d = 0; else d = 1; if(d != ex->r) { deltaa = n * (ex->r - d) * ex->a; deltab = n * (ex->r - d) * ex->b; } else { deltaa = 0; deltab = 0; } w->wa = w->wa + deltaa; w->wb = w->wb + deltab; } </pre></div><!–ENDSCRIPT–> <!–STARTSCRIPT–><!–source lang="cpp"–><div class="source"><pre> //training.h #include "main.h" struct exercise { double a; // first operand double b; // second operand double r; // expected result }; struct weight { double wa; // weight a double wb; // weight b }; double wrand(); int getexercise(struct exercise *ex, struct weight *w, int n); void learning(struct exercise *ex, struct weight *w); </pre></div><!–ENDSCRIPT–> If anyone has some hints it would be very appreaciated… Thanx, fadeh <!–EDIT–><!–/EDIT–>[Edited by - fadeh on August 26, 2005 3:29:10 AM]<!–EDIT–><!–/EDIT–>

S41NT

122

August 26, 2005 07:37 AM

It's been a while since I worked with ANNs, but if I'm not mistaken, it's possibly not a coding problem.

You'll never get it working with that threshold. You either add a bias to the node (you know, an input which is allways 1: d = x1*w1 + x2*w2 + 1*w3) or change the threshold to something positive (I would make the neuron biased).

If I'm wrong in my assumptions, I'm sure someone here will correct me.

S41NT

122

August 26, 2005 07:50 AM

It just occurred to be maybe I should try to explain. I'm not very good at explaining but let's try:

You want d to be non-positive if the answer is false (A AND B = false)

So that means A * w1 + B * w2 must be equal or less than zero. Because it has to be less than zero for A = 0 and B = 1 or A = 1 and B = 0, both w1 and w2 will have to be non-positive (do the math. None of them can be positive because 1 and 0 must return the same answer as 0 and 1).

So we need w1 and w2 <=0. How will we ever get a positive d? It's not possible.

Now suppose you have a bias. Then you can get an answer with something like

w1 = 0.6; w2 = 0.6; w3 = -1;

fadeh

Author

122

August 26, 2005 01:18 PM

Many thanx man, now it works perfectly (adding a bias as u explained)... but another question went out: why doesn't work with xor? :D
If i had understood what you said the things don't work because:
0 XOR 1 = 1, 1 XOR 0 = 1, so wa > 0, wb > 0 but in that case the things don't work with 1 XOR 1 = 0... so if wc < 0 and |wc| > |wa + wb| the things are done for 1 XOR 1 but messed up for 0 XOR 1, 1 XOR 0... so let's choose wa < 0, wb < 0, wc > 1... this way 0 XOR 1, 1 XOR 0, XOR 1 are ok, but 0 XOR 0 no.... so it seems that the bias in this case doesn't work or i missing up something...?

Thanx

fadeh