Hi all... I am currently doing some simulations of fotball playing. The setup i have is two teams of agents trying to develop good strategies for fotball playing. Currently all players have full knowledge about the field through a vector of player structures, containing the players x and y coordinates on the field as well as a fitness value of that player indicating how good its position is (i.e. how many opponents are surrounding it, how close to the goal etc. (subject to changes here)). The vector also contains the balls position. Now, for the reinforcement learning part i will have each player have a neural network to approximate the target function which is of the type FIELD STATE -> real number. That is, it takes the current state of the field and values it to some value. In order to train the network i need in each time step choose a set of states for each player to check which gives the best next state. I cant check all of them since there are way too many, and my problem thus comes down to choosing a good set of states to test..?! So far i have thought of taking the obvious state where each player and the ball just continues in the direction of their speed vectors. Also, the two states where the ballkeeper passes the ball to another player or shoots the ball can be included. Now, we can also include a pass and/or shot in all directions. This gives very many states all of a sudden... To do the professional way i guess you would choose the state which yeilds the highest entropy. Can this be calulated in this kind of setting? Does anybody have any ideas or suggestion how to choose the states to test? Also links to similar projects would be appreciated.. thank you for your time --Spencer "Relax, this dragon is sleeping..."

Definitely check out the Robocup software competition and the associated publications.

Ultimately, you have stumbled upon one of the biggest problems with reinforcement learning in realistic domains: the problem of training the system on a sufficiently broad spectrum of samples from the domain so as to correctly classify the decision boundaries within that domain. Unfortunately, there is no easy answer.

You''re going to have to read a lot of literature to get a feeling for the different methods that people have developed for handling this sort of problem and then determine which one is most appropriate for your scenario.

Good luck,

Timkin

thanks for the answer Timkin.
I will look in to it right away


--Spencer

"Relax, this dragon is sleeping..."

Here''s something I found which you may find useful:
KaBaGe-RL: Kanerva-based Generalisation and Reinforcement Learning for Possession Football

If you''re interested in learning more about reinforcement learning then you should be aware of the Reinforcement Learning Repository at the University of Massachusetts, Amherst.

By far the best introductory book available is Sutton and Barto''s Reinforcement Learning: An Introduction. You don''t need to buy the book because the online version is pratically identical (although it''s always nice to have a hard copy). There is also some code available on their website which accompanies the book.