basic neural network

nir143 · 2011-09-27T22:53:24

[font=Arial,]after reading some articles about neural network(back-propagation) i try to write a simple neural network by myself. ive decided XOR neural-network, my problem is when i am trying to train the network, if i use only one example to train the network,lets say 1,1,0(as input1,input2,targetOutput). after 500 trains +- the network answer 0.05. but if im trying more then one example (lets say 2 different or all the 4 possibilities) the network aims to 0.5 as output i searched in google for my mistakes with no results :S ill try to give as much details as i can to help find what wrong: -ive tried networks with 2,2,1 and 2,4,1 (inputlayer,hiddenlayer,outputlayer). -the output for every neural defined by: [color=#00008B] double[color=#000000] input [color=#000000] =[color=#000000] [color=#800000] 0.0[color=#000000] ;[color=#000000] [color=#00008B] for[color=#000000] [color=#000000] ([color=#00008B] int[color=#000000] n [color=#000000] =[color=#000000] [color=#800000] 0[color=#000000] ;[color=#000000] n [color=#000000] <[color=#000000] layers[color=#000000] [[color=#000000] i[color=#000000] ].[color=#2B91AF] Count[color=#000000] ;[color=#000000] n[color=#000000] ++)[color=#000000] input [color=#000000] +=[color=#000000] layers[color=#000000] [[color=#000000] i[color=#000000] ][[color=#000000] n[color=#000000] ].[color=#2B91AF] Output[color=#000000] [color=#000000] *[color=#000000] weights[color=#000000] [[color=#000000] n[color=#000000] ];[color=#000000] while 'i' is the current layer and weight are all the weights from the previous layer. -the last layer(output layer) error is defined by: [color=#000000] value[color=#000000] *([color=#800000] 1[color=#000000] -[color=#000000] value[color=#000000] )*([color=#000000] targetvalue[color=#000000] -[color=#000000] value[color=#000000] );[color=#000000] while 'value' is the neural output and 'targetvalue' is the target output for the current neural. -the error for the others neurals define by: [color=#00008B] foreach[color=#000000] neural [color=#00008B] in[color=#000000] the nextlayer sum[color=#000000] +=[color=#000000] neural[color=#000000] .[color=#000000] value[color=#000000] *[color=#000000] currentneural[color=#000000] .[color=#000000] weights[color=#000000] [[color=#000000] neural[color=#000000] ];[color=#000000] -all the weights in the network are adapt by this formula(the weight from neural -> neural 2) [color=#000000] weight[color=#000000] +=[color=#2B91AF] LearnRate[color=#000000] *[color=#000000] neural[color=#000000] .[color=#000000] myvalue[color=#000000] *[color=#000000] neural2[color=#000000] .[color=#000000] error[color=#000000] ;[color=#000000] while LearnRate is the nework learning rate(defined 0.25 at my network). -the biasweight for each neural is defined by: [color=#000000] bias[color=#000000] +=[color=#2B91AF] LearnRate[color=#000000] *[color=#000000] neural[color=#000000] .[color=#000000] myerror[color=#000000] *[color=#000000] neural[color=#000000] .[color=#2B91AF] Bias[color=#000000] ;[color=#000000] bias is const value=1. that pretty much all i can detail, as i said the output aim to be 0.5 with different training examples thank you very very much for your help . [/font]

Artificial Intelligence Programming

Started by nir143 September 23, 2011 01:48 AM

12 comments, last by AlanMcCormick 13 years, 2 months ago

nir143

Author

100

September 25, 2011 06:51 PM

This tutorial may help you: http://www.codeproje...recipes/BP.aspx

the formulas are same as i wrote here\in my program:S

FrankyRP

143

September 26, 2011 10:36 AM

Your output layer error seems different. (I didn't check the other formulas).
Did you try to run the tutorial's code?

http://francky.me/

nir143

Author

100

September 26, 2011 01:45 PM

yea,ive just forget to write a line where :
[color="#1C2837"][color="#00008B"]foreach[color="#000000"] neural [color="#00008B"]in[color="#000000"] the nextlayer
sum[color="#000000"]+=[color="#000000"]neural[color="#000000"].[color="#000000"]value[color="#000000"]*[color="#000000"]currentneural[color="#000000"].[color="#000000"]weights[color="#000000"][[color="#000000"]neural[color="#000000"]];
[color="#1C2837"][color="#000000"]
[color="#1C2837"][color="#000000"]myerror=sum*myoutput*(1-myoutput);
[color="#1C2837"][color="#000000"]
[color="#1C2837"][color="#000000"]its seems like when im training the normal 4 examples,
each 2 example with different output just contradict each other(change the weights in opposite directions) :S

[color="#1C2837"][color="#000000"]

AlanMcCormick

100

September 27, 2011 10:53 PM

I 'm not sure about your code either, but have you tried reducing your learning rate to something much smaller? 0.1, 0.05 etc. If your learning rate is too high it may not converge

basic neural network

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

basic neural network

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines