Do someone knows what reinforcement learning, Q-Learning, Markov Decision Process and SARSA are about? Basically I'm doing some researches on ANN to control a NPC AI. I could use a backpropagation net to make my NPC learn, but this technique is not aimed to be used in real time processes. I'm looking for a way to allow the NPC learn in real time. I have read some articles dealing with "Reinforcement Learning". I think I could use the reinforcement learning with ANN. cheers!! [Edited by - jolyqr on June 8, 2006 12:52:23 PM]

Quote: Original post by jolyqr
Basically I'm doing some researches on ANN to control a NPC AI.

What attributes in the NPCs state space are you trying to control? What sort of control problem are you wanting to solve (regulation, compensation, etc.)? Without first specifying the control problem, its very hard to talk about learning a control function (so called "intelligent control").

Quote:
I'm looking for a way to allow the NPC learn in real time.

This is a significant and difficult open research problem. Online learning of control functions is extremely problematic, even more so when you don't have a supervisory control system to learn from.

Quote:
I have read some articles dealing with "Reinforcement Learning".
I think I could use the reinforcement learning with ANN.

If you want to learn the parameters of an ANN while using it for a given task, then you want to solve a dual estimation problem. Off the top of my head, I think Eric Wan did some research in this area. If that's a dead end, let me know and I'll dig through my bibliography file for a specific reference.

Cheers,

Timkin

Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…

I really don’t know a lot about reinforcement learning, so I don’t know if I need to do some “regulations” or “compensations”…

I know this is a very difficult area. My supervisor told me it’s possible to do some reinforcement learning in ANN. I have taken a look on Internet, I did not find a lot on using a ANN in reinforcement learning. The only examples I found use q-learning…

Anyway, if you have any information concerning that matter, don’t hesitate to give it to me…

cheers

Reinforcement Learning: An Introduction
Richard S. Sutton and Andrew G. Barto

online version :
http://www.cs.ualberta.ca/~sutton/book/ebook/the-book.html

Regards
Michael

Quote: Original post by jolyqr
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…

The typical approach to this sort of problem is to break down the agent's behaviours into those that can be handled by a single AI methodology and then to combine them into your final agent. For example, your agent might make use of a pathfinder for deciding how to get to spatial goals, a planner for choosing those goals, a reactionary system for moving in combat, a decision system for choosing targets in combat, a sensory system for determining the world context, etc, etc. One or more of these systems may be in operation at any time and they will be changing the state of the agent accordingly.

If you want to investigate the use of reinforcement learning for your game agent, I would suggest that you choose one problem (such as target selection) and concentrate on that. To attempt to create a complete agent that learns all of its behaviours as mimicry of a human expert would be an exceedingly large task (and indeed, would probably be achieved by combining several smaller learning systems).

Cheers,

Timkin

Quote: Original post by Timkin

Quote: Original post by jolyqr
Theoretically, the problem I want to solve is not too complicated. I would like my NPC to fire the player, to look for items and to flee…

The typical approach to this sort of problem is to break down the agent's behaviours into those that can be handled by a single AI methodology and then to combine them into your final agent. For example, your agent might make use of a pathfinder for deciding how to get to spatial goals, a planner for choosing those goals, a reactionary system for moving in combat, a decision system for choosing targets in combat, a sensory system for determining the world context, etc, etc. One or more of these systems may be in operation at any time and they will be changing the state of the agent accordingly.

If you want to investigate the use of reinforcement learning for your game agent, I would suggest that you choose one problem (such as target selection) and concentrate on that. To attempt to create a complete agent that learns all of its behaviours as mimicry of a human expert would be an exceedingly large task (and indeed, would probably be achieved by combining several smaller learning systems).

Cheers,

Timkin

ok. cheers