So I have been reading up on AI lately, and from what I can tell ANN''s are what I need (recognizing trends and predicting based on a large dataset). My biggest question, which from the tutorial or two that I read was not mentioned, is the actual TOPOLOGY of the network. Pretty much the generic explanation is "input layer -> hidden layer -> output". What exactly is this? How do I design a network, how do I know which neurons should link to which? What does adding more neurons do to the network, and what does refactoring the network topology (just changing links) do? I am going to google a bit, but I figured I would post and see if anyone can explain some of this, or link me to a GOOD tutorial. Thanks

The most common topology is a feed-forwad multilayer perceptron. The output from each neuron in layer i is connected to every input of layer i+1, and the operations of the network reduce to some matrix algebra.

Other topologies can be used, and this area isn''t understood very well. With a static topology, backpropagation is a common method to train the neural network. If the topology is allowed to change, then a different learning algorithm would probably be needed. Some algorithms can even add and remove neurons during learning as more data is received. The simplest algorithms just use a fixed topology and adjust weights. There are also algorithms that use recursive neural networks.

A simple example:

let''s say that for the given problem, there are 2 input variables, and one desired output. The input layer could have 2 nodes, the output layer would have one node, and we''ll use one hidden layer with 4 nodes. Each layer will be fully connectedd to the next layer.

Between the input layer and the hidden layer, there are 8 links. Each input is connected to all 4 of the hidden neurons. Between the hidden and output layers there are 4 links, each hidden neuron connects to the output neuron.

The hidden neurons should use some kind of sigmoid function. The output neuron can just use summation.

Then with some sample data, backpropagation can be used to try to train the network by adjusting weights.

Note that neural networks generally aren''t very good at extrapolation. Near the values that the network was trained with, it will generate a fairly smooth function. But outside of those values, the neural network''s output can be completely useless.

quote:

---

Original post by The Reindeer Effect
So I have been reading up on AI lately, and from what I can tell ANN''s are what I need (recognizing trends and predicting based on a large dataset).

---

Aaaarrrrrggggggggggghhhhhhhhhhhhhhhhhhhh!!!! *tears out some more hair*

Why is this the common conclusion that people come to when looking at data mining??? Did the popular media of the 80s so deeply pervert the perception of AI that the only thing people see when they look at AI is ANNs?

The tasks you describe are sub-problems within data mining. ANNs are certainly NOT the only tool for this task and typically speaking, aren''t the best tool for the task either! Don''t limit your capabilities by restricting your toolbox to just one tool. Learn about other techniques: regression (linear and nonlinear), causal modelling, Bayesian Inference, clustering/classification, etc. Once you''ve got these tools in your toolbox, you''ll be in a good position to try different tools on the job and find the one that works the best for the specific problem you have... and typically speaking, evaluating best is difficult... different techniques will generally give you subtley different results that you''ll have difficulty identifying the ''correct'' result within (because typically the correct result is inaccessible... or you wouldn''t need to do the data mining in the first place)!

Timkin

Well timkin beat me to it, but I can add a little. What you have is indeed a data mining problem, so my AI professors homepage may have some valuable information for you (data mining is his specialty).

Try this out:
Statistical Data Mining Tutorials

While it is a lot of reading, it contains tutorials for most of the tools you would need in your "toolbox" for data mining, including neural nets. I recommend reading the Decision Trees tutorial first, as it is fairly simple and might possibly be good enough for your application.

quote:

---

Original post by Timkin
Why is this the common conclusion that people come to when looking at data mining??? Did the popular media of the 80s so deeply pervert the perception of AI that the only thing people see when they look at AI is ANNs?

---

Well, gosh, Timkin, doesn''t AI stand for Artificial Intelligence? And isn''t a neural network really just an exact computer replica of a human brain, intelligence and everything? seems like that would be the perfect way to do EVERYTHING in AI EVER!

---

"Sneftel is correct, if rather vulgar." --Flarelocke

quote:

---

Original post by Sneftel

quote:

---

Original post by Timkin
Well, gosh, Timkin, doesn''t AI stand for Artificial Intelligence? And isn''t a neural network really just an exact computer replica of a human brain, intelligence and everything? seems like that would be the perfect way to do EVERYTHING in AI EVER!

---

A neural network may be close to the human brain, but not an ARTIFICIAL neural network (ANN) ANNs are a very stripped down version of the way we think the brain works. Saying that they are the exact same thing isn''t accurate IMHO.

In any case, he didn''t say ANNs aren''t AI. He said that for data mining, there are many tools, but people only seem to consider ANNs a lot of the time.

<SPAN CLASS=smallfont>quote:

---

Original post by The Reindeer Effect
So I have been reading up on AI lately, and from what I can tell ANN''s are what I need (recognizing trends and predicting based on a large dataset). My biggest question, which from the tutorial or two that I read was not mentioned, is the actual TOPOLOGY of the network. Pretty much the generic explanation is "input layer -> hidden layer -> output". What exactly is this? How do I design a network, how do I know which neurons should link to which? What does adding more neurons do to the network, and what does refactoring the network topology (just changing links) do?

---

</SPAN>

The most commonly implemented artificial neural networks are multilayer perceptrons (MLP). Typically, the input "layer" is simply the model inputs (the independent variables). The hidden and output layers are collections of nodes, which calculate weighted sums of the values coming from the previous layer and run them through (usually) sigmoid transfer functions. These nonlinear nodes are essentially basis functions which are assembled to produce an approximation to the training data. Most often (by far), each node receives data from all nodes in the previous layer, in addition to a bias value (equivalent to the intercept in a linear regression). Sometimes, "jump" connections are included, which allow information to flow between non-contiguous layers. Also, some implementations utilize only partial connections between layers. Increasing the number of hidden nodes increases the maximum possible complexity of the function being approximated (which may lead to overfitting if not used carefully).

I recommend at least browsing through the Usenet comp.ai.neural-nets FAQ, which one may find here:

ftp://ftp.sas.com/pub/neural/FAQ.html

Additionally, I suggest "Computer Systems That Learn", by Weiss and Kulikowski, as an excellent introduction to important issues surround theuse of neural networks and other empirical modeling algorithms.


-Predictor
Data Mining in MATLAB


[Edited by - Predictor on February 13, 2009 4:06:33 PM]

my goodness. What does it take to close a <sarcasm> tag around here?

---

"Sneftel is correct, if rather vulgar." --Flarelocke

<SPAN CLASS=smallfont>quote:

---

Original post by Timkin
The tasks you describe are sub-problems within data mining. ANNs are certainly NOT the only tool for this task and typically speaking, aren't the best tool for the task either!

---

</SPAN>

Do you mean that neural networks are never the best tools for such tasks, that neural networks are usually not the best or that neural networks are not always the best, or something else altogether?

-Predictor
Data Mining in MATLAB




[Edited by - Predictor on February 13, 2009 4:11:58 PM]

quote:

---

Original post by Predictor
Do you mean that neural networks are never the best tools for such tasks, or that neural networks are not always the best, or something else altogether?

---

I think my AI professor put it best: "ANNs are the second best way to do anything". It''s a useful techique, and the generality with which it may be applied is very attractive. But it''s a technique that cannot exploit the invariants and heuristics of a particular problem, and thus is usually doomed to be less effective than a custom-engineered AI solution.

---

"Sneftel is correct, if rather vulgar." --Flarelocke