I am a grad. student at MIT finishing up a thesis (just a few more days now!) on AI learning methods and games. It is an examination of the past and present of AI learning methods in games, and some thoughts about the future. I was wondering what some of you might think on some of the stuff I tried to discuss in my thesis...My main question is: "What do you think is going to be the future of AI learning methods in games? And, how should people begin to think about using AI learning techniques in games?" I'd be very curious to hear what you folks might think. I've listed a few more specific questions down below, to give people a better/more specific idea of what I am wondering about. Thanks in advance! Christian Baekkelund (draco at mit dot edu) P.S., Due to the cross-disciplinary nature of these questions, I posted this here to the tech. forum, to try to get more of a feel on what people think with regards to the technical issues, but I also made a similar post on the game design forum to try to get at the design issues. Not just blindly spamming everything... :) ----------------- If anyone wants a bit more clarification on what I'm thinking about or have a bit more time, here's some more specific additional questions: - In addition to "the future role of learning methods in games", where and how have past attempts at using AI learning techniques in games has succeeded or failed? - Technically, are there any learning methods in particular that folks think have a lot of potential (or that are particularly unuseful)? - Also, is there any specific technical research that folks think needs to happen to make better learning methods for games? - Is there a particular type of game or genre of games that the use of some learning methods would especially benefit from? - What new genres of games do folks think would come about with better and easier use of AI learning techniques in games?

Some way for creating, storing and updating patterns from previous experience would be very very good. Especially if it learns quickly and can find non-obvious patterns in the board states.

Also, being able to simplify board states with previous knoweledge would be very useful.

Being able to use terrain to find things that it needs, especially non-obvious ones would be useful. ie. learning that a good hiding spot is not where everyone can shoot at you. And definatly not, if your standing next to a couple of tons of explosives.

From,
Nice coder

Quote:

Original post by Nice Coder Some way for creating, storing and updating patterns from previous experience would be very very good. Especially if it learns quickly and can find non-obvious patterns in the board states.

Exactly, but specifically, what kinds of patterns?

Quote:

Also, being able to simplify board states with previous knoweledge would be very useful.

"board states"?...do you just mean the general current game state?

Quote:

Being able to use terrain to find things that it needs, especially non-obvious ones would be useful. ie. learning that a good hiding spot is not where everyone can shoot at you. And definatly not, if your standing next to a couple of tons of explosives. From, Nice coder

Definitely a good idea, but then the question is: is this not scriptable? Or are there special advantages gained over just writing a script like:

if (distanceToExplosives < 10.0) then move to new spot

etc...You could do it either way...but is one better than the other?

It is scriptable.
And with board states, i meant states on a bord game, like zerts or chess, or checkers.

With patterns, just things that happen.
Being able to differenciate between different situations, so that they can find the different characteristics, which lets them predict what they should do.

You could have things like having one learn that running from the back, and blowing the player up with a hand grenade, which has been already depinned. That would require some thinking, and some logic, and some characteristic learning (like when you pull the pin off a hand-grenade it goes BOOM! also, when behind someone they cannot see you. when someone doesn't see you, they can't shoot you or run away).

Things like that.

From,
Nice coder

Crosspost

Quote:

Original post by ChristianBaekkelund Exactly, but specifically, what kinds of patterns?

Ideally you could let the program figure that out... just give it the ability to find, remember, and utilize patterns it finds, and then use a genetic algorithm to let the pattern recognizers evolve. This should serve to weed out the useless patterns, and the ones that are effective will rise to the top.

Games have all kinds of patterns that we humans can pick up on... some consciously, others unconsciously. If we try to hard code in the pattern recognition then we're limiting ourselves, and the intelligent agents, to the patterns we know we recognize vs. all that are really there.

Quote:

"board states"?...do you just mean the general current game state?

If I'm off base here Nice Coder let me know, but I assume he was saying:

Yes, a "board state" would just be the current state of pieces(players, objects, etc...) on the board, their health, location... basically anything at all about the current state of the game. He said it would be nice to be able to "simplify board states with previous knowledge", which would help with various search related tasks. For example, when you're in state "A" your agent doesn't even need to worry about actions X, Y, or Z... which can let you trim branches off search trees and save time and processor cycles and all that good stuff.

Quote:

Definitely a good idea, but then the question is: is this not scriptable? Or are there special advantages gained over just writing a script like: if (distanceToExplosives < 10.0) then move to new spot etc...You could do it either way...but is one better than the other?

For many games a script is enough. Most games only seek to "simulate intelligence", and not really recreate it in any academic sense. If I, as a player of your game, see the opponents working together, running when they're hurt or over-matched, and my frame rates are fine, then the game AI coder has done their job, whatever their method.

That said, I would personally prefer a solution that did not involve scripting. Scripting is a very explicit way of doing it, and if you don't think of every option, or script an action for every possibility, then nothing will happen when those possibilities are realized. It is a very reflexive way to generate actions such that if situation X arises, action Y will be taken every single time. Many gamers hate that and do not want such predictable opponents. Some randomization can help with this, but can also lead to an action being chosen that makes no sense at all for the situation, which a gamer will also hate.

One advantage to scripting is that it is very fast to execute, and you get the behavior you specify and not some weird combination of actions that could potentially crash the game. If you see the agent does not act as you'd like in a certain circumstance, then you simply find the "If" statement for that circumstance and modify. If it were coded with another method then that behavior may have emerged out of the interaction between a dozen underlying factors, and fixing that could be quite an ordeal.

Another advantage scripting has is that it does not need to be trained or have any data preloaded. Many of the more "organic" methods involve getting a starting data-set and then running thousands or millions of training sessions on it, or doing a great deal of in-game training and testing.

Please note: I am not an AI researcher, or in the AI business (yet), so take what I've said with a grain of salt. I have simply read dozens of AI books, had a single undergrad class in AI, and am quite interested in the subject. Speaking of which, was it tough to get into MIT's AI program? Has it been as tough and rewarding as one might imagine MIT's AI program would be? :)

Wyrd-One

Quote:

Original post by Fruny Crosspost

I think he explained why he did that.