Hi, I've recently completed an article/tutorial on Artificial Neural Networks and I'd like some feedback on it. Here's the link Is it interesting? Easy to follow? Do you have any recommendations? Thanks.

Alright, but dont take the criticism badly please :)

You begin your tutorial with the thing that annoys me most with ANN: You try to make people believe they work like the brain works. They dont. Throwing biological references can only discredit your work.

Quote:

"since pretty much anything can be represented by binary, the only real limit to what you can do with ANNs is the speed of your processor and the amount of memory your computer has."

Is this true? I thought ANN could not perfectly represent certain classes, like XOR.

Quote:

There is another type of ANN that's called sigmoid something...

If you mention it in a tutorial, learn what it is first. Anyhow, anyone doing programming should learn enough math to know what a sigmoid is.


Also, Ive looked at your "line smoothing" application. It looks a lot like you are building a Bezier curve from the points. I thought you would like to know the term.

Agree on the biological reference point. It can be helpful as an analogy, but...

Quote:

"since pretty much anything can be represented by binary, the only real limit to what you can do with ANNs is the speed of your processor and the amount of memory your computer has."

You don't really make any mention here of the decision surface, which is related to the number of hidden layers in the network, and affects the type of problems the NN can learn. 0-hidden-layer networks have a linear surface and thus can only "solve" linear problems; et cetera. I'm not enough of an AI buff to know for sure, but I'm fairly certain this places extensive theoretical and practical restrictions on your assertion that what you can do with NNs is limited to processing power.

NNs are not real brains and cannot be used to solve particularly complex, multi branching problems. They ultimately just make decisions based on inputs, and are really pretty useless as you scale the number of inputs -- selecting a good, small input set and using an NN as part of a larger group of solutions is more effective than trying to throw more inputs and layers at the thing and assuming it will just "figure it out." They try to match patterns. And let's not forget how difficult they can be to train.

Quote:

If you mention it in a tutorial, learn what it is first. Anyhow, anyone doing programming should learn enough math to know what a sigmoid is.

Furthermore, it's somewhat incorrect to say the network is a "sigmoid something ANN;" the sigmoid function in relation to neural networks refers to the function used as the transfer function.

That said, it is pretty easy to read, although I question the lack of math, since neural networks are pretty math heavy and I've found that really understanding the math helps you figure out when they'd be useful and when they're just a waste.

Quote:

Is this true? I thought ANN could not perfectly represent certain classes, like XOR.

A given perceptron (node in the network, essentially) cannot solve XOR (a single perceptron can solve fewer problems than it cannot solve, in fact). However, a network of multiple perceptrons can solve XOR (XOR is the "hello world" litmus test for your backpropagation routine; that is, if you can teach a newborn network XOR via backpropagation, you're in decent shape so far).

My questions:
Why do you need to write a tutorial about artificial neural networks?

Why is your tutorial better than the thousands of tutorials and books already out there?

If you're doing it just for personal satisfaction, no problem... but then why do you need feedback on it?

If you're doing it to show the world that you know something about artificial neural networks... does it matter if they know? (cyberspace <==> anonymity)

I know I'm sounding very pessimistic today... but that's because I am pessimistic today... if you're going to write something, research it thoroughly first. Make sure you understand all of the concepts intimately and can demonstrate them all before sitting down to write. Then place yourself in the mind of the reader and decide how someone with little or no knowledge is going to be able to make use of what you write.

Doing this will prevent you from churning out what is just another page hyping up ANNs with little or no justification.

Regards,

Timkin

I think there is a good way to research something: to write all about it and then show result to get feedback on it. It's good because includes an emotional constituent. Someone needs to read books to get knowledge, but someone could feel bored doing it. And feedback information shows spaces in one's knowledge as against of books. Either methods, Analysis and Synthesis are good to get any certain information. It's true because human brain "was disigned" for both ways to reach any knowledge.

Quote:

Original post by Timkin My questions: Why do you need to write a tutorial about artificial neural networks?

I like writing about what I know.

Quote:

Why is your tutorial better than the thousands of tutorials and books already out there?

It depends on the target audience. In my case, since I'm not an expert on ANNs, I couldn't possibly make an advanced tutorial. That is actually a good thing in my opinion because I haven't found a decent tutorial to get people started with ANNs; all those that I've seen so far tend to be too complicated for newbs.

Quote:

If you're doing it just for personal satisfaction, no problem... but then why do you need feedback on it?

I want to know if I'm good at writing tutorials and if I should continue. Also, I wanted people to point out my weaknesses in doing so.

Quote:

If you're doing it to show the world that you know something about artificial neural networks... does it matter if they know? (cyberspace <==> anonymity)

The main reason why I did this was because I enjoy writing down what I know ,and, also, I wanted to help other people. I guess it's true though, It's nice to be recognized for something and, in my opinion, it really doesn't matter if people don't know who I am or not, It's better to have them be impressed by the work and not by the person behind it.

Quote:

I know I'm sounding very pessimistic today... but that's because I am pessimistic today... if you're going to write something, research it thoroughly first. Make sure you understand all of the concepts intimately and can demonstrate them all before sitting down to write. Then place yourself in the mind of the reader and decide how someone with little or no knowledge is going to be able to make use of what you write. Doing this will prevent you from churning out what is just another page hyping up ANNs with little or no justification.

My tutorial is accurate as far as I know. I studied theory on ANNs for a whole semester and I had some of my papers handy while writing the tutorial. Also, I managed to implement an ANN from scratch using this methodology so I know that the main concepts were true for certain.

----

Quote:

Original post by jpetrie Agree on the biological reference point. It can be helpful as an analogy, but... "since pretty much anything can be represented by binary, the only real limit to what you can do with ANNs is the speed of your processor and the amount of memory your computer has."

Well, I guess that I thought that since you can have a single binary output from one specific ANN, then why wouldn't you be able to group many different ANNs together(each trained to recognize different patterns) and find a way to combine their outputs to get a more accurate output. What I mean by that is that, if you wanted your ANN to recognize the letter 'B', you could fuse two different ANNs to give you a single output. One ANN could have been trained to recognize curves and the other could recognize straight, vertical lines... So if you're analyzing a bitmap and there are curves present in it but no straight lines, the first ANN will output 1 while the second will output 0... Because it's an 'AND' situation; the final output of this group of ANNs would be 0. It's only if both curves and a straight line are present that a 1 will be returned. Of course, in this case, many shapes could involve curves and a straight line so it probably won't be accurate all the time, but I guess it gives the machine some sense of 'reasoning' as to why the final output is what it is.
Of course the statement I made was a personal opinion and not fact.

Quote:

You don't really make any mention here of the decision surface, which is related to the number of hidden layers in the network, and affects the type of problems the NN can learn. 0-hidden-layer networks have a linear surface and thus can only "solve" linear problems; et cetera. I'm not enough of an AI buff to know for sure, but I'm fairly certain this places extensive theoretical and practical restrictions on your assertion that what you can do with NNs is limited to processing power.

It's a beginner's tutorial, I' don't to confuse anyone.
I guess I just can't see a limit to what ANNs can do... I mean, they can assist in speech recognition; that's pretty convincing. Also, I was being a bit futuristic on that statement because I assume that, given current trends, we will find more ways to construct ANNs and some of these methods will be highly effective.

[Edited by - Flashthinker on July 24, 2007 7:28:58 PM]

Quote:

Original post by Flashthinker Of course the statement I made was a personal opinion and not fact. ... I guess I just can't see a limit to what ANNs can do... I mean, they can assist in speech recognition; that's pretty convincing. Also, I was being a bit futuristic on that statement because I assume that, given current trends, we will find more ways to construct ANNs and some of these methods will be highly effective.

Thats the heart of the problem. As you are not, will all due respect, very knowledgable about ANN *and* the field of machine learning, your tutorial ends up propagating misconceptions about ANN common among people who studied them for a little while. Chief among those misconceptions:

1- ANN work just like the brain!!!
2- ANN can potentially do anything!!!

This end up doing more harm than good.

...slightly OT...

Quote:

since pretty much anything can be represented by binary

What is π (pi) as a binary string please? ;)

Quote:

Original post by Flashthinker I want to know if I'm good at writing tutorials and if I should continue. Also, I wanted people to point out my weaknesses in doing so.

It's a blog, it has too little images, it lacks java 4-6 applets, it's too short, the next part is still missing, it lacks pseudocode. Properly designed web sites are more readable.

Quote:

Original post by Timkin ...slightly OT... Quote: since pretty much anything can be represented by binary What is π (pi) as a binary string please? ;)

π = 0000001111000000