Advertisement

Howto learn AI

Started by January 15, 2007 11:33 AM
26 comments, last by gentrinity 17 years, 10 months ago
You have one huge problem to solve. There's an absolutely massive number of variables to track, amd you'd have to identify which small handful of those variables contributed to success, and which were irrelevant. To explain, lets take some examples from the footie AI we are looking at here.

In the run-up to a goal, each player would have chosen many actions, say three or four in the five seconds leading up to the goal. That's 30+ actions to assess. You can't just say the actions of the player with the ball - other players would have been maneuvering for position, keeping defenders occupied and other useful things. So it isn't even just those 30 actions you've looking at - it's the order of the actions, the positions of your team-mates and opposition players at the time. You quickly end up with thousands of possible combinations that might describe a successful play, and you have to search that huge search-space for the patterns that consistently lead to success. That sort of data mining is hard to do off-line. Finding a way to do it reliably in real-time would guarantee you fame and glory forever.
Hey, your idea seems like it could work "in theory".

Your best bet is to start small and build up your functionality until it does what you expect. You have to ask yourself how you are going to get your rules working in the game and how you will deal with probability.

Here is a nice tutorial on FSM's : http://www.ai-junkie.com/architecture/state_driven/tut_state1.html
Advertisement
Incidentally, as a simple example of WHY you have to use if/then logic in something like this... the player, at the very minimum, has to ask itself "do I have the ball?" There are far more after that. If/then logic is almost unavoidable. You can cloak it in a neural network, if you like. Really, all you do is tell the NN to learn for itself that, at the very core, it needs to recognize whether it has the ball or not and then decide other things from there. So you are insisting that the NPC learn that for itself rather than give it an if/then for the obvious state of "ballcarrier?". Just something to ponder.

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

Hey yall,
@shadowisadog
I read the article about FSMs, and this really looks like I could use it. This is how I pictured it to work.
However is there a way to let the agent learn which state it is supposed to use? I thought of a way like this:
First I need a function that "gives sugar or punishes", meaning it tells an agent whether he did something good or bad.
Lets create an easy function like:

function A: The closer the ball the the opponents goal is the better.

However this might make him dribble and run with the ball all the time. So there should be a function like:

function B: The closer the ball in a certain amount of time to the opponents
goal is the better.

Function B might cause the agent to pass the ball to a different player who might have more space infront of him to run along or even pass to an other player.
Another function could look something like this:

Function C: If you shoot and score you and the agents that assisted in the making of the goal will get sugar en masse.

Creating those functions and of cause function that punish the agents like an intercepted pass etc. could tweak the descision making process somehow. So that each agent according to their skill (e.g. passing shooting etc) will descide what is best for the team.

But of cause there must also exist some "world rules" that descide whether or not the pass goes where it is supposed to go or the dribbling is successfull. It also regulates the agents stamina etc. So the agents dont just have to learn how to act in certain situations but also how to behave in those "world rules".

I think if I start with an easy setup lets say 3 on 3 or 5 on 5 game and simple states like pass, shoot or run it might be possible to create something unique (according to the world rules). I know that it is hardly possible to create a simulation of a real soccer game, but that is as i said before not my goal.

@InnocuousFox
I know that I wont get arount progging if/then logics. What I ment when i said that, is that i dont want an AI that I program all by myself meaning using if/then logics. Since I dont know what the optimal behavior in my world rules are. I want those agents to find the optimal or at least an optimal solution to that problem.

I think the best way for me to proceed is to just program a couple of states and see how the AI handles it. But since this is a lot of work, I would still appreciate what you more expreienced AI-programers think about my idea. If it is possible to make it work. Or where I might face serious problems.

Thanks in advance
cherio Woltan

The main problem you face with agents that can learn on their own is tranining the agents. It seems like you are really looking for some kind of neural network but they are kind of just buzz words and I am sure there are other ways to approach your problem.

Apparently your goal is to make a simulation that will discover the best strategy by itself rather then have you tell it what to do. As you might have gussed this is dramatically increasing the amount of work you will have to do, as well as the complexity of the system.

I recommend reading up on neural networks and genetic algorithms, but I also recommend reading up on some of the other methods of machine learning AI's as well. I don't have a ton of experience in the machine learning field so I am sure someone will be able to help you better then I can ;).

Dont forget:


The Closer to My goal the ball is, AND not controlled by My player -- the worse it is.

Similarly: The Closer to the Opponents goal the ball is AND is Controlled by My players -- the better...


The problems with the if-thens is that they have alot of clauses with ANDs/ORs/NOTs in them to spell out specific situations.



Trying to trace what factors lead up to the success requires alot of processing
because it can trace back thru alot of previous actions and results.

Another problem is defining 'success' and 'failure' and most games have more than one flavor for particular parts of the game. Example-- its not just making Goals, but also preventing your opponent from making goals.

And for the parts of the game which are not close to either absolute event that defines success/failure there are partial results (like keeping control of the ball and advancing towards your opponent's goal).
--------------------------------------------[size="1"]Ratings are Opinion, not Fact
Advertisement
Quote: Original post by wodinoneeye
The problems with the if-thens is that they have alot of clauses with ANDs/ORs/NOTs in them to spell out specific situations.
[snip]
And for the parts of the game which are not close to either absolute event that defines success/failure there are partial results (like keeping control of the ball and advancing towards your opponent's goal).

That's why you don't use if/then to necessarily analyze the situation but rather use accumulators in various perceptions and concepts to analyze the situation and then throw the results at a fuzzy decision engine to decide the appropriate action for that moment.

For example, you may have accumulators for the position on the field, how pressured you are at the moment, how open various teammates are, etc. Then by looking at the constellation those various accumulators are in, you can decide,
"should I pass," "should I dribble," "should I back up" etc.

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

Hey yall,
thanks again for the feedback.
I must say the more I think about the AI the less I think it is going to work.
1. I am a complete noob when it comes to AI
2. It is too complex to find functions that evaluate position of 22 players, even if it is on a 2D map.
3. Shooting goals or preventing the opponent team from makeing goals is just too simplifyed, since there is a whole bunch of action befor that, that lead to the success.

ERGO:
I am going to do a different approach. And once again you have the "honor" to tell me what you think. (I hope this thread is not considered to be spam and a waste of bytes ;)

I am fixing to program the AI of a certain position on the field (e.g. defender). This would reduce the number of situations quite a bit. The next thing is to reduce the total number of players. Like that I think I should be able to formulate some success functions. It is also easier to test.

My question for this posting:
I have now read articles that deal with FSM, Neural Nets and genetic algorithems. I can picture my AI to work with a FSM including some neural nets that alter their constants with a gentic algorithm. Is this a usual approach? Or should I maybe take a look at some other methods?

Thanks again for your help
cheiro Woltan
Quote: Original post by Woltan
I have now read articles that deal with FSM, Neural Nets and genetic algorithems. I can picture my AI to work with a FSM including some neural nets that alter their constants with a gentic algorithm. Is this a usual approach? Or should I maybe take a look at some other methods?

For now, you should forget you have ever heard of NNs or GAs. NNs work really well for pattern recognition, but not as much for decisions such as the ones that your players will be making. At least not on a "noob" level. What you are going to want is to look into decision trees. Your decision trees will be based on a lot of FSMs and other accumulator values. Again, never mind NNs and GAs until such time as you learn other sorts of AI techniques.

Oh.. and did you mean for the subject to be "...teach AI"? as in "How do I teach an AI?" Or "How do I make a learning AI?" That changes things a bit.

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

Woltan: Try these terms:

expert systems
rules-based programming


For games that involve personality and creativity, the academic subjects aren't going to be as useful to you as just making it work. The most popular method is the iterative approach, where you get your AI working early and play against it often. Then you can begin identifying and correcting circumstances where the computer opponent is weak, or could react in a more interesting way. The tools and algorithms will become more clear to you as you gain experience developing your games, and may also give you some surprising inspirations about how the game itself can be improved. Think of it more as "(creatively) applied intelligence". ;)

Remember games are about creating interesting choices for the players, not necessarily finding the most optimal path to victory.

This topic is closed to new replies.

Advertisement