After reading the book On Intelligence by Jeff Hawkins, I got pretty interested in AI/Neural Networks. I've scoured the web pretty heavily in search of tutorials and example code, and I've came up with quite a bit. I've put together my own fully customizable class covering the basic network, using the Backprop learning algorithm. Everything works perfectly fine, and I understand it pretty well. My problem lies in how hard it was for me to get some pseudocode that didn't involve complex math formulas as the explanation. It seems that straightforward information doesn't really exist for the mathematically challenged like myself (I'm great at math, I just haven't taken any classes beyond algebra, lol). I'd really like to get a book or two (maybe more) that go into recurrent networks, reinforcement learning, unsupervised learning, content-addressable memory, etc. I'm just worried that I'll end up buying something way over my head. Does anyone have any suggestions? Perhaps I'd be better off asking this community for tutorials on these subjects? I really liked Neural Netware by André LaMothe in the NN section here on gamedev, it just didn't go as far as I'd like.

For a relatively easy explanation of some alternative neural architectures (CPM, RCE, PNN/GRNN, SDM, etc.), I suggest "Advanced Methods in Neural Computing", by Wasserman (ISBN-13: 978-0442004613).

Thanks for the suggestion, I've got it on order.

In case you read this reply, does it go into recurrent networks or genetic algorithms much? I'm thinking my interest is going to lie heavily with those two areas for quite a while before I move on to any other topics. My knowledge is still very limited on these subjects, so please forgive any nonsense I might supply.

I might need to just spend some time learning how to properly read mathematical notation.. (or whatever the proper terminology is).

Genetic algorithms is a different field to artificial neural networks, so you won't tend to see them covered by a text on the latter.

This is what I started on NN with:

http://www.generation5.org/articles.asp?Action=List&Topic=Neural+Networks

Quote: Original post by Mythics
I'd really like to get a book or two (maybe more) that go into recurrent networks, reinforcement learning, unsupervised learning, content-addressable memory, etc. I'm just worried that I'll end up buying something way over my head.

Holy jesus on a scone, you rock. It is downright rare to see someone actually interested in a rigorous, practical approach to NNs in the context of machine learning, as opposed to "i like NNs cuz their like branes how can i make my NN more like a brane". Hooray for you!

I personally don't have any direct recommendations for machine learning textbooks with a NN focus... my graduate course didn't use a textbook. As a generalized AI text, Russell and Norvig is the gold standard. Some people bash it for using its own slightly idiosyncratic pseudocode rather than giving everything in C, but I can't fault it for that, and the stuff it gives is accessible without much math knowledge (just don't expect to use it as a copy-paste source). However, while it does go into NNs, it isn't a machine learning textbook, and only goes so far in that area. For stuff that really drills down into machine learning theory, I really like the look of this book, though that's based fully on looking at the table of contents.

Thanks a lot for the references and info. I'm glad to get a bit of positive feedback. :)

Regarding: "Advanced Methods in Neural Computing", by Wasserman

Quote: Original post by Mythics
Thanks for the suggestion, I've got it on order.

In case you read this reply, does it go into recurrent networks or genetic algorithms much? I'm thinking my interest is going to lie heavily with those two areas for quite a while before I move on to any other topics. My knowledge is still very limited on these subjects, so please forgive any nonsense I might supply.

I might need to just spend some time learning how to properly read mathematical notation.. (or whatever the proper terminology is).

"Advanced Methods in Neural Computing" includes an entire chapter on genetic algorithms, and briefly ties them in with SDMs ("sparse distributed memories").

This text does not, however, cover recurrent neural networks. Generally, those are used for time-series data, and I suspect that most practitioners use more conventional neural networks with lagged variables or other pre-processing methods.

For a good high level coverage of machine learning it's hard to go past Tom Mitchell's book "Maching Learning" (McGraw Hill, 1997). It's aimed at upper undergraduate and beginning post-graduate students, but it does not pre-suppose knowledge of AI or statistics. So rather than getting into the nitty gritty of techniques and mathematics, it gives a higher level exposition of the methodology and importantly, the connections between the techniques.

I second Timkin's recommendation of Tom Mitchell's "Machine Learning" book. I've used it for my undergraduate course in Machine Learning and it does provide a very good chapter on ANN's.

Amazon Link