basic neural network
September 23, 2011 01:48 AM
[font=Arial,]
after reading some articles about neural network(back-propagation) i try to write a simple neural network by myself.
ive decided XOR neural-network, my problem is when i am trying to train the network, if i use only one example to train the network,lets say 1,1,0(as input1,input2,targetOutput). after 500 trains +- the network answer 0.05. but if im trying more then one example (lets say 2 different or all the 4 possibilities) the network aims to 0.5 as output
i searched in google for my mistakes with no results :S ill try to give as much details as i can to help find what wrong:-ive tried networks with 2,2,1 and 2,4,1 (inputlayer,hiddenlayer,outputlayer).
-the output for every neural defined by:
while 'i' is the current layer and weight are all the weights from the previous layer.
-the last layer(output layer) error is defined by:
while 'value' is the neural output and 'targetvalue' is the target output for the current neural.
-the error for the others neurals define by:
the nextlayer
-all the weights in the network are adapt by this formula(the weight from neural -> neural 2)
while LearnRate is the nework learning rate(defined 0.25 at my network). -the biasweight for each neural is defined by:
bias is const value=1.
that pretty much all i can detail, as i said the output aim to be 0.5 with different training examples
thank you very very much for your help
.
21,607
September 23, 2011 06:51 PM
Debugging your problem is hard to do (especially without access to the code itself), and I am not willing to do it for you. But I can tell you how I would go about doing it.
Do you understand why the update rules are what they are? They are supposed to be taking one little step in gradient descent. The specific formulas you use are ways of computing the derivative of the error function (typically the square of the difference between the output and the desired output) with respect to each weight. You can try to change the weights a tiny bit, measure the change in the error function and see if that matches your computation.
Do you understand why the update rules are what they are? They are supposed to be taking one little step in gradient descent. The specific formulas you use are ways of computing the derivative of the error function (typically the square of the difference between the output and the desired output) with respect to each weight. You can try to change the weights a tiny bit, measure the change in the error function and see if that matches your computation.
September 23, 2011 08:22 PM
ive try some tests,as i said if im using only one training example everything works fine(atleast aim to 0.04 mistake after 600 trains).
i dont understand what you advice me to do,
change the weights little bit?:S(that created randomally),i think i misunderstood you,
the 4-5 formulas i worte here are ok fine right?
tyvm for your help:)
i dont understand what you advice me to do,
change the weights little bit?:S(that created randomally),i think i misunderstood you,
the 4-5 formulas i worte here are ok fine right?
tyvm for your help:)
21,607
September 23, 2011 08:34 PM
ive try some tests,as i said if im using only one training example everything works fine(atleast aim to 0.04 mistake after 600 trains).
i dont understand what you advice me to do,
change the weights little bit?:S(that created randomally),i think i misunderstood you,
the 4-5 formulas i worte here are ok fine right?
tyvm for your help:)
This is what I mean by changing the weights a little bit. After you evaluate the network, pick a weight, add 0.001 to it and evaluate it again. If you consider the error function as a function of the value of the weight, you can now compute (f(w+0.001)-f(w))/0.001 , which should be close to the derivative. I don't know if the formulas you posted are correct, but the way I would figure out if they are correct is by trying to understand them as a gradient-descent step, which involves computing that derivative.
September 23, 2011 09:07 PM
im not strong in this kind of math:S
but ive followed few guides about the formulas and about the theory of course and as i checked the derivative function is correct,
i try to give as much details as i can to make the formulas\code understandable:)
I went through the code few times and still dont understand where the problem:S
but ive followed few guides about the formulas and about the theory of course and as i checked the derivative function is correct,
i try to give as much details as i can to make the formulas\code understandable:)
I went through the code few times and still dont understand where the problem:S
September 24, 2011 02:33 AM
ive try also to use genetic algorithm instead of back-propagation and they output aim to be 0.5:S
that means the problem should be in the feed-forward formulas?
that means the problem should be in the feed-forward formulas?
160
September 24, 2011 03:49 PM
Try using 0.1 instead of 0, and 0.9 instead of 1, for your inputs and outputs. What is your activation function?Also, try removing the bias nodes for now to see if that makes a difference.
September 24, 2011 04:15 PM
im using logistic activation function-sigmoid curve.
ive try what you said with 0.9\0.1 instead of 1\0 and removing the bias but the output still aim to be 0.5:S
ive try what you said with 0.9\0.1 instead of 1\0 and removing the bias but the output still aim to be 0.5:S
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
