ve done some stuff with genetic algorithms, entities evolve over generations. Now I want to move on to neural networks. But all the tutorials Ive found so far uses a combination of AG and NN's. Where the NN's is encoded binary and lives in generations, the best in the generation moves on to the next, natural selection and mutation. But I dont want to do the generation based things anymore. A single agent, or agents, that evolve over time and isnt killed and replaced by a new generation seems much more interesting and challenging. Anyone got any documentations / suggestions / inspiration or pointers on how to achieve this on the run learning? I guess that it shouldnt be as much different as before. You have to set a time (like 60 seconds) and everytime a cycle is complete every agents.. hmm... ive lost the word... the number that tells how well it performed, sort of a score. Anyways, every 60 seconds you calculate how well each agent did and edit thier weights according to the normal new weight = actual outout - wanted output. Two questions arise here. I dont know that exact formula either (hehe looks like Ive lost much tonight :P, I would look it up but my favorite site generation5 is down it seems), that which calculates the new weight. Should I run that individually for all the neurons in the net? Cause it calculates only on the output? And what about if I dont even know the wanted output? I read articles but some things is, obviously, hard to grasp. A few points are still there before I can breach the bubble with my first project. Anyone can comment the problems I stated?

Hi there!
If you know what the output should be (let's call it target output), and know the actual output and input for the network, you can train it with the Backprop algorithm. Google it, or there should be some tutorials over generation5, also I can recommend AI Techniques for Game Programmers by Mat Buckland and also AI Application Programming by M. Tim Jones.
If you don't know what the target output should be, you need do to something with Unsupervised learning, which uses genetic algorithms and scoring (the word you wrote was correct, agents usually return a score of how well they did). Where you have some initial genes, go through the network, and use somekind of score system the get some idea of how well the genes did. Then mutate the best genes, and do this until the fitness score is below a given value. Sounds easy, but working out the fitness function and working out the number of neurons on each level is the actual art of it. :)
But be aware that you possibly need thousands(if not hounderds of thousands) generations before the network is going to do well (depends).
Hope this helps!

Quote:
I guess that it shouldnt be as much different as before. You have to set a time (like 60 seconds) and everytime a cycle is complete every agents.. hmm... ive lost the word... the number that tells how well it performed, sort of a score.

I believe you're talking about the "fitness value"

Quote: And what about if I dont even know the wanted output?

Then what are you doing??, 'cause if you don't know the wanted output, how can you optimize something????

You should know it...., at least, that's how I see it....

And as for not using GA...., the other method I read about is "Backpropagation", but as I understand, it requires much interaction with the user in order to train the net..., when GA training doesn't....

Well, I hope I made some sense...

PS: I'm a newbie to this area too, so don't believe everything I say....[wink]

Yes the "Fitness value" was the word I was loocking for.

And the rule I had in mind of how to change the weights is called the "Delta rule":

Quote:
The Delta Rule

It is fairly self-evident that training the perceptron requires modifying the weights. The Delta rule is a simple learning rule that states the weights should be adjusted by the difference between the desired output and the actual output.

And what did confuse me was how to apply this to a NN with many neurons. However, when I reread the tutorial I saw that further down it was stated that the Delta rule wasnt applicable on those kind of nets.

Quote:
Training multilayer perceptrons/neural networks is not a simple task. There are many ways, such as back-propagation, but all these methods are non-trivial and require special attention.

And you suggested back-propagation, Ill look deeper into that.

haora: Yes sometimes you doesnt know the wanted output. For example if youre trying to simulate some kind of artificial life, the part of AI that Im interested in, you need to be able to handle the situation whitout a fixed "goal". However you still implent stuff as score or fitness value, that could be compared to the pleasure/pain systems of normal biological agents. But thats still a bit advanced for my level. :P

Generation5 stated that training a multilayer, or multineuron NN, was very hard. And the back-propagation system required "special attention". I was more aiming for something of a unsupervised system, that can run on its own and me just watching how it evolves. Anyone know of such a system?

Quote:
Generation5 stated that training a multilayer, or multineuron NN, was very hard. And the back-propagation system required "special attention". I was more aiming for something of a unsupervised system, that can run on its own and me just watching how it evolves. Anyone know of such a system?

Yes, you can use GA to train the NN without any supervision..., there's a great introduction tutorial in www.ai-junkie.com

Quote:
es sometimes you doesnt know the wanted output. For example if youre trying to simulate some kind of artificial life, the part of AI that Im interested in, you need to be able to handle the situation whitout a fixed "goal". However you still implent stuff as score or fitness value, that could be compared to the pleasure/pain systems of normal biological agents. But thats still a bit advanced for my level. :P

Ok, I thought you were talking about not knowing anything about the output...., I get it now...., I'm actually trying to do some artificial life things my self....[wink]

Backpropagation is an unsupervised algorithm for training your ANN. It will of course depend on your problem, but to begin with a regular feed-forward network with backpropagation is a good place to start. The math is rather intuitive and there's been extensive research and applications with such systems. The special attention referred to for backprop networks I guess applies to the different parameters for your simulation. Initially you will tweak at least 3-4 different parameters to get acceptable results and as the system gets more advanced the number of parameters (and the time spent tweaking) is likely to increase, but you are probably familiar with this from GAs :)

The training of the network itself does not require human attention though.

This is a rather nice introductionary text to backpropagation networks.

haora: Me and another guy is thinking of starting a project together. You seem to be pretty much on the same level, why dont you join? If you want to talk my email adress is in my profile. :)

silmaril: That link was wonderful. :) That will keep me busy for a while. Perhaps back-propagation is what Im looking for.

Quote: Original post by silmaril
Backpropagation is an unsupervised algorithm for training your ANN.

Backprop is a supervised training approach, because you know the inputs and the target outputs for the network.

Quote: Original post by dande

Quote: Original post by silmaril
Backpropagation is an unsupervised algorithm for training your ANN.

Backprop is a supervised training approach, because you know the inputs and the target outputs for the network.

Indeed it is. I got caught up with explaining that it didn't require online interaction with a human and forgot that supervised/unsupervised is actually technical terms used to describe machine learning techniques. My bad :)