Having problems with neural network training C#

Metalsoul · 2010-12-12T08:06:40

Hi everyone!I'm new and I've a problem with a neural network I'm working on.I'm writing it in c#I think the problem is in the "train" method.Could anyone help me?I'm going to paste here my whole code so you can read it.Thanks in advice!(Sorry for the bad english) public class NeuralNetwork { /// <summary> /// Create a new NeuralNetwork object /// </summary> /// <param name="inputs">Number of input nodes</param> /// <param name="outputs">Number of output nodes</param> /// <param name="hiddens">Number of hidden nodes</param> /// <param name="Beta">Indicates the learning rate, from 0.0 to 1.0, best is 0.5</param> public NeuralNetwork(int inputs,int outputs,int hiddens,float Beta)//Constructor/initialization { X = new float[inputs]; Y = new float[outputs,2]; W1 = new float[inputs * hiddens]; // first serie of synapses H1 = new float[hiddens,2]; W2 = new float[hiddens * hiddens]; // second serie of synapses H2 = new float[hiddens,2]; W3 = new float[hiddens * hiddens]; // third serie of synapses H3 = new float[hiddens,2]; W4 = new float[hiddens * outputs]; // third serie of synapses errors = new float[(outputs*hiddens)*2]; B = Beta; Eta = (inputs / outputs) * 0.5f;// quickprop/rprop reset(false); } //Variable Declaration private float[] X; //Input vector private float[,] Y; //Output vector private float[,] H1; //Hidden vector private float[,] H2; //Hidden vector private float[,] H3; //Hidden vector private float[] W1; //Synapse vector/s 1 private float[] W2; //Synapse vector/s 2 | =Four hidden levels private float[] W3; //Synapse vector/s 3 / private float[] W4; //Synapse vector/s 4 / public float[] errors; private float B; private float Eta; /// <summary> /// Reset the neural network so it can be trained again /// </summary> /// <param name="kill">Reset to 0 all the threshold</param> public void reset(bool kill) { int value = 1; if (kill) value = 0; //For the initialization we will use the "Equal Distribution" random method, from -0.5 to 0.5? //first serie for (int a = 0; a < W1.Length; a++) { W1[a] = Game1.RandomBetween(0.0f, 0.5f) * value; } //hiddens 1 for (int a = 0; a < H1.Length/2; a++) { H1[a, 1] = Game1.RandomBetween(0.0f, 0.5f) * value; } //second serie for (int a = 0; a < W2.Length; a++) { W2[a] = Game1.RandomBetween(0.0f, 0.5f) * value; } //hiddens 2 for (int a = 0; a < H2.Length/2; a++) { H2[a,1] = Game1.RandomBetween(0.0f, 0.5f) * value; } //third serie for (int a = 0; a < W3.Length; a++) { W3[a] = Game1.RandomBetween(0.0f, 0.5f) * value; } //hiddens 2 for (int a = 0; a < H3.Length / 2; a++) { H3[a,1] = Game1.RandomBetween(0.0f, 0.5f) * value; } //third serie for (int a = 0; a < W4.Length; a++) { W4[a] = Game1.RandomBetween(0.0f, 0.5f) * value; } //outputs for (int a = 0; a < Y.Length/2; a++) { Y[a, 1] = Game1.RandomBetween(0.0f, 0.5f) * value; } } //Activation function private float sigma(float val) { return (float)(Math.Tanh(B * val)); } //Training /// <summary> /// Train the NeuralNetwork with the specified Training Set /// </summary> /// <param name="TS">used Training Set</param> public int train(float[,,] TS) { /* * The training set should be built so: * float[proofs,test level,value] * so for example * we should execute a test aiming to get the NeuralNetwork pointing to the maximum number among four * TS=new float[100,2,4] * so 100 proofs * level 0 contains input data * level 1 contains optimal output * with 4 input and output */ int returner = 0; //input initialization float[] inputs = new float[Y.Length/2]; //training sets cycle for (int a = 0; a < TS.GetLength(0); a++) { for (int b = 0; b < Y.Length/2; b++) { inputs = TS[a, 0, b]; } //update.. returner = update(inputs); //########### BACK-PROPAGATION ########### //calculate errors for (int c = 0; c < Y.Length / 2; c++) { errors[c] = TS[a, 1, c] - Y[c, 0]; } //last level (Y) for (int c = 0; c < Y.Length/2; c++) { Y[c, 1] = (float)(errors[c] * B * (1 - Math.Pow(Y[c, 0], 2))); } //########### I THINK THAT ERRORS OCCURS HERE ########### //level H3 for (int c = 0; c < H3.Length / 2; c++) { float sum = 0; //first sum all the values for (int d = 0; d < Y.Length/2; d++) { sum += W4[(d * (H3.Length / 2)) + c] * Y[d,1]; } //then change the values for (int d = 0; d < Y.Length/2; d++) { H3[c, 1] = (float)(B * (1 - Math.Pow(Y[d,0], 2)) * sum); } } //level H2 for (int c = 0; c < H2.Length / 2; c++) { float sum = 0; //first sum all the values for (int d = 0; d < H3.Length / 2; d++) { sum += W3[(d * (H2.Length / 2)) + c] * H3[d, 1]; } //then change the values for (int d = 0; d < H3.Length / 2; d++) { H2[c, 1] = (float)(B * (1 - Math.Pow(H3[d, 0], 2)) * sum); } } //level H1 for (int c = 0; c < H1.Length / 2; c++) { float sum = 0; //first sum all the values for (int d = 0; d < H2.Length / 2; d++) { sum += W2[(d * (H1.Length / 2)) + c] * H2[d, 1]; } //then change the values for (int d = 0; d < H2.Length / 2; d++) { H1[c, 1] = (float)(B * (1 - Math.Pow(H2[d, 0], 2)) * sum); } } float val = 0; //level W4 for (int c = 0; c < H3.Length / 2; c++) { for (int d = 0; d < Y.Length/2; d++) { val += H3[c, 1] * Y[d,1]; } } for (int c = 0; c < H3.Length / 2; c++) { for (int d = 0; d < Y.Length/2; d++) { W4[(c * (Y.Length/2)) + d] += -Eta * val; } } //level W3 val = 0; for (int c = 0; c < H2.Length / 2; c++) { for (int d = 0; d < H3.Length / 2; d++) { val += H2[c, 1] * H3[d, 1]; } } for (int c = 0; c < H2.Length / 2; c++) { for (int d = 0; d < H3.Length / 2; d++) { W3[(c * (H3.Length / 2)) + d] += -Eta * val; } } //level W2 val = 0; for (int c = 0; c < H1.Length / 2; c++) { for (int d = 0; d < H2.Length / 2; d++) { val += H1[c, 1] * H2[d, 1]; } } for (int c = 0; c < H1.Length / 2; c++) { for (int d = 0; d < H2.Length / 2; d++) { W2[(c * (H2.Length / 2)) + d] += -Eta * val; } } //level W1 val = 0; for (int c = 0; c < X.Length; c++) { for (int d = 0; d < H1.Length / 2; d++) { val += X[c] * H1[d, 1]; } } for (int c = 0; c < X.Length; c++) { for (int d = 0; d < H1.Length / 2; d++) { W1[(c * (H1.Length / 2)) + d] += -Eta * val; } } } return returner; } //Give the input /// <summary> /// Update the NN by giving it the inputs. /// </summary> /// <param name="inputs">Values to pass.</param> /// <param name="training">Is this a training session?.</param> /// <param name="expected">Only if training is true: the array value you expect from the input.</param> /// <returns>Returns the index of one of the outputs.</returns> public int update(float[] inputs) // inputs->X { X = inputs; //first transmission for (int a = 0; a < H1.Length / 2; a++) { H1[a,0] = 0; for (int b = 0; b < X.Length; b++) { H1[a,0] += X * W1[(a * X.Length) + b]; } H1[a, 0] = sigma(H1[a, 0] - H1[a, 1]); } //ok //second transimssion for (int a = 0; a < H2.Length / 2; a++) { H2[a,0] = 0; for (int b = 0; b < H1.Length/2; b++) { H2[a,0] += H1[b,0] * W2[(a * H1.Length / 2) + b]; } H2[a, 0] = sigma(H2[a, 0] - H2[a, 1]); } //ok //third transimssion for (int a = 0; a < H3.Length / 2; a++) { H3[a,0] = 0; for (int b = 0; b < H2.Length / 2; b++) { H3[a,0] += H2[b,0] * W3[(a * H2.Length / 2) + b]; } H3[a, 0] = sigma(H3[a, 0] - H3[a, 1]); } //last sigma, output results float max = 0; int highest = 0; for (int a = 0; a < Y.Length/2; a++) { Y[a,0] = 0; for (int b = 0; b < H3.Length/2; b++) { Y[a,0] += H3[b,0] * W4[(a * H3.Length / 2) + b]; } Y[a,0] = sigma(Y[a,0]-Y[a,1]); //also find the highest if (Y[a,0] > max) { max = Y[a,0]; highest = a; } } //done return highest; }}[edit: added source tags][Edited by - InnocuousFox on November 19, 2010 1:10:28 PM]

Artificial Intelligence Programming

Started by Metalsoul November 19, 2010 08:59 AM

22 comments, last by sjaakiejj 14 years, 2 months ago

sjaakiejj

130

December 10, 2010 10:46 AM

Hmm it seems as though you are correct, the link seems to be down today. Try and access it again sometime later, it should be back up again in no-time.

Good luck!

Metalsoul

Author

100

December 11, 2010 08:42 AM

Thank you very much!
And emm.. could you help me with one more problem?
Would you explain me weights update?
Expecially the delta? Thank you soo much!

Metalsoul

Author

100

December 12, 2010 07:04 AM

I made it!
I succesfully created a neural network of type feedforward with backpropagation, the only main difference with the starting one, is that the neurons don't have thresholds, only weights get modified.

sjaakiejj

130

December 12, 2010 08:06 AM

Oh, I didn't notice in your code that you put a threshold on your Neurons. A Neural Network with thresholds cannot be trained, so that would be why it didn't work.

Nice to hear you got it working though.

Having problems with neural network training C#

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Having problems with neural network training C#

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines