Im looking for a simple tutorial on neural nets, specifically the backprop algorithm, everything Ive found on the net is full of complex equations that I simply don't understand. My aim is to write a backprop library in C so I'll need something that explains all the error stuff clearly. I've already written the feed-forward phase, but Im having trouble understanding the back prop phase, so any nice tutorials would be greatly appreciated. Noel

The backprop algorithm isn't that bad, but does use a lot of math like you pointed out.

Unfortunatly there is no single formula, although there are several common ones.


As easily as I can describe it, you would compute the outputs of every node, then compute the error of the output. Use the delta rule (which is a bit of math beyond a simple gamedev post) on the outer layer to compute the error. On the inner nodes, you need to use a slighly modified delta rule that takes the weighted sums of the other nodes into account to find their error. Rinse and repeat. With the error in hand, adjust the weights as needed.

There is no ideal set of rules to use on the inner layers for computing the error or for adjusting the weights. Common values for computing the error (IIRC) are linear functions, sigmoids, and scaled sums. For adjusting the weights, you can use momentum or not, per-element updates or batch updates, or other add in other factors to help you reach your ideal congruence point.

I'm sure there are hundreds of different methods for computing the internal error, since there are lots of research papers coming out all the time, so I doubt there is a single 'good' tutorial that covers all the options available.

Neural Network formulas are intimidating, I know. However, it doesn't seem like something you can avoid if you wish to use them. I have trouble understanding them myself for much the same reason.

It's worth crunching through the neural net stuff at least once, to get a feel for it, and then write a simple application (text recognition is always a good start) to see it in action. You can probably google some universitie's assigned projects and collect sample data.

I found a great tutorial on NeuralNets @ www.planet-source-code.com
(its for VB)
See/Download Tuturial on Planet-Source-Code.com

I wrote a C++ app for a very simple text recognition (recognizes '5', '6', '7', '8', '9' or something like that), complete with learning data and test data. I can email it to those interested, if I can find it on my homedir dump DVD. I don't want to post it, cause I know some loser who needs to write an assignment will google for it and use it, which would cause me to hunt him down and toss him into a woodchipper :-). But lemme know if interested.

Neural networks are not a trivial thing and the equations involved can get quite nasty. There's really no way to avoid this, you are probably best just getting a good text on the subject and reading it over and over until it makes sense.