Hi! I've had an idea for a project in my AI course! A game entity in a virtual environment, perhaps with two controls: it turns left or right (all the while moving forwards). The goal would be to maximise the distance travelled or minimise the number of obstacles hit... something like that. So hopefully the character would walk around, learning by itself to avoid obstacles. Can anyone recommend any types of neural network which could be suitable for this application? And other words I should google or resources to look at for more info? Reading through wikipedia, it seems reinforcement learning is what I'm looking for (rather than unsupervised learning??). But I can't find much resources on specific ANN types to use with this, or any practical examples. (I also wonder if this is a feasible project for someone with no practical experience of neural networks. Bearing in mind I have lots of time to learn)

I don't know if it is in the scope of what you're doing, but you should check out Ken Stanley's NEAT program: http://www.cs.ucf.edu/~kstanley/

Quote: Original post by xDan
Can anyone recommend any types of neural network which could be suitable for this application? And other words I should google or resources to look at for more info?

Why have you chosen to use a NN classifier rather than, say, a Naive Bayes, k-means, or SVM classifier?

>> Why have you chosen to use a NN classifier rather than, say, a Naive Bayes, k-means, or SVM classifier?

Because NNs are exciting. The Terminator used them did he not?

@kirkd: that seems to be evolving them, but still interesting!

Indeed NEAT is an evolutionary computation methodology. Interesting stuff that might give a little inspiration. 8^)

Ah, I didn't realise evolution of them was considered reinforcement learning (just found that out through wikipedia).. I may well end up evolving them though as there seems to be more (understandable!) info around about that. And I have wanted to try that. So thanks :)

(by reinforcement I was thinking more if there was any way to have the networks learn during use... i.e. a single entity that moves around improving in intelligence rather than spawning many generations of them. in a more somewhat-biologically-realistic manner.)

If i were you I'd look at another technique, NNs have their strength and weaknesses. I personally wouldn't use them as a controller for an entity. Especially since one outlying piece of data could potentially undo all previous learning or even prevent learning from occurring, not too mention the training would be quite difficult, and once trained the neural network would just simulate a large state machine:

on these three inputs do this, on another three inputs do that, etc... you're better off creating a weighted state machine.
If you're dead set on using NNs take a look at my blog for explanations and source code.

Yes, this simple problem obviously doesn't really require AI at all. I may modify the problem. But the main reason is to get experience with neural networks. I find them interesting. Also my course is part robotics, this could be considered a simulated robot as well as a game entity.

I have seen similar things which seem to have worked, e.g. http://www.ai-junkie.com/ann/evolved/nnt1.html

if you want experience with NNs rather do something like a classification/optimization problem and not a control problem, thats what NNs are really good at...

I also want to do something FUN. With little virtual vehicles rolling around :)

So unless NNs actually *cannot* do this (but it would appear they can, see that link I posted, and also I believe there is a GameDev article on a similar thing, but with supervised learning), then it is what I will do!