Quote: Original post by willh

Quote: Original post by InnocuousFox

Quote: Original post by willh
We disagree on the reason for ML. I use ML techniques to solve problems that I am unable to solve using my own facilities of reason.

Nifty, I admit, but not really relevant to game AI, however.

Dave, if you're not going to contribute something constructive then please refrain from commenting. Thank you.

Pointing out that something is either not relevant or not applicable to the problem at hand is certainly constructive. Given that this forum is a game AI forum, and that you were offering examples of AI that have very few analogs in the game AI realm, I was pointing out that those examples were not, in your word, "constructive." Actually, given the diminished relevancy of those particular examples, I could have said something snide to you along the lines of "if you're not going to contribute something constructive." However, as I just said, separating the proverbial wheat from the chaff, is, indeed, constructive. Between the two of us, we did as much.

Quote: Original post by willh

Quote: Original post by InnocuousFox

Quote: Modeling player behavior in MMORPGs so they can 'persist' even when offline

But you can do much of this simply by tweaking knobs in reinforcement learning a la the Creature in Black & White. If you keep it out of the black box, you can also allow the player to do this by hand or by selecting preset combinations of data.

Perhaps, but what I've seen so far is something that is a poor and unbelievable approximation at best. I don't see how one would call that 'problem solved' by any measure. The last time I played an MMORPG I did not have the option of 'persisting' in a realistic way. EDIT: And my boxer in Fight Night for the PS3 does NOT play anything like me when the computer controls it, despite the few controls I have to tweak.

I agree... no one has done it well, if at all. However, that is mostly a design decision in that players have expressed being somewhat creeped out by the notion of their avatars continuing on without them. To them, that isn't "playing" the game... it is training a bot. Also, no matter how well the bot mimic's play, the players will not perceive it that way. If something negative happens to that avatar, they will come in with the belief that it wouldn't have happened if they had been around to run the show. This is more psychology than technology. Unfortunately, no matter how good our technology game development is ultimately driven by player psychology. More industrial and academic AI problems often do not face this barrier.

This really reinforces the point that, in the larger sense, game AI is not about solving a problem as it is entertaining the player. Soren Johnson (Lead Designer and AI programmer on Civ 4) covered this issue well in his GDC 2008 talk, "

">Playing to Lose: AI and Civilization". You can view a version of it he gave at a Google Tech Talk at the above link.

Quote: Original post by willh

Quote: Original post by InnocuousFox

Quote: Original post by willhWalMarts problem of knowing when and how to move stock between stores isn't that much different than managing units in an RTS.

This actually speaks to my point earlier. the WalMart problem is an NP-Hard optimization problem that has a "right answer". RTS games played against humans don't really have a "right answer".

I don't understand the reasoning behind your argument.

Are you suggesting that the real world, with its weather, traffic, currency exchange rates, variable fuel rates, and shifting regional/national purchasing trends is somehow more predictable/easier than an RTS? If you've got that solved then you're wasting your time on games and should be dealing with commodity trading. There might even be a nice job opening for you on Wall Street (or within WalMart!). ;)

I didn't suggest that I had it solved nor did I suggest that it is easier than an RTS. I'm saying that, at a certain level, the problems are inherently different. While you could say that they are both optimization problems, there is a "maximization point" that the WalMart problem can approach. If that's what you are trying to accomplish in the AI for an RTS, you are missing the point. Again, see my comments above and Soren's video.

Oh... and yes, I have done NP-Hard mathematical optimization algos -- including bin-packing, the vehicle routing problem, the knapsack problem, vertex cover, etc. Both in industry and in game development. Thanks for the vote of confidence, though.

Quote: Original post by InnocuousFox
Pointing out that something is either not relevant or not applicable to the problem at hand is certainly constructive. Given that this forum is a game AI forum, and that you were offering examples of AI that have very few analogs in the game AI realm, I was pointing out that those examples were not, in your word, "constructive."

Here we go...

This topic is about Machine Learning algorithms AND Game AI. What I said seems ESPECIALLY relevant given the context of Kevins article. You should know that though, since you started it.

Quote: Original post by InnocuousFox
Interesting... thought that there would be more comments on this article either on here or on the site itself.

Anyone with decent reading and comprehension skills would take this to mean that your intent in posting was to have memebers of this board make comments. Well here I am, regretting it, and beginning to appreciate why there wern't many other comments.

Quote: Original post by InnocuousFox
Actually, given the diminished relevancy of those particular examples, I could have said something snide

You just did say something snide.

Quote: Original post by InnocuousFox
I agree... no one has done it well, if at all. However, that is mostly a design decision

That is a 'Red Herring' argument. Rather than argue against the original claim, you're attempting to change the topic.

Quote: Original post by InnocuousFox
This really reinforces the point that, in the larger sense, game AI is not about solving a problem as it is entertaining the player.

You're confusing cause and effect.

Entertaining the player is the problem to be solved. Therefore game AI is about solving a problem. Much like a house is meant to be lived in, but not the hammer that built it.

Quote: Original post by InnocuousFox
Soren Johnson (Lead Designer and AI programmer on Civ 4) covered this issue well in his GDC 2008 talk

You're making an 'Appeal to authority'.

Quote: Original post by InnocuousFox
I didn't suggest that I had it solved nor did I suggest that it is easier than an RTS. I'm saying that, at a certain level, the problems are inherently different. While you could say that they are both optimization problems, there is a "maximization point" that the WalMart problem can approach.

They are optimization problems. One is a simplified version of the other, so should be much easier to solve.. Funny how it isn't though. Hmmm..

Quote: Original post by InnocuousFox
Oh... and yes, I have done NP-Hard mathematical optimization algos -- including bin-packing, the vehicle routing problem, the knapsack problem, vertex cover, etc. Both in industry and in game development. Thanks for the vote of confidence, though.

I don't question your intelligence Dave; you really do seem like a smart guy. I think you're rude and that you act like a bully though.

I'm done with this thread too. Surprise.

Thanks Kevin Dill for your posts and your article. I enjoyed reading it and found your comments insightful and educational.

One last thing Dave-- Debate.org. The topic will be about machine learning not being relevant to game AI. You won't have your mod hammer to swing around, but you are an expert whose written a book so should be able to hold your own. The winner will be determined by the votes of lay people based on the merits of the arguments.

I could see ML used in an 'offline' fashion in MMORPGs to correct NPC behaviors/situational factors in specific 'quest' scenarios (frequently ones that every player in the game goes thru, existing/reused constantly for years). Players reactions/solution and goal/subgoal achievment speed (metrics) would be recorded to correlate with what the NPCs/props did. The goal being to make the scenario challenging/interesting enough by adjusting the scenario either automaticly (hard -especially when script logic is involved) or better as a visualization/sumarization tool for manual adjustment(much much simpler).

The CPU use by ML can definitely be prohibitive realtime and simply generating logs of scenarios played out by the players would limit the inworld server impact. The logs would then be subject to the external processing/statistical analysis/factoring and then be used to generate changes to the scenarrio. Iterative 'batch' cycles would eventually grow the scenario's settings within some optimal range (and could later again allow adjustment if the game mechanics change).

It would be useful just as a tool identifying a player experience 'problem' (too easy/to hard/loopholes by certain character classes or skillsets/etc..) so that a human could decide how to tweak NPC control logic/scenario choreography (static changes or put in place some simple automatic adaptive mechanism that would incrementally adjust a few factors identified as a 'fix' by the human gamemaster).

The above is more like intelligent filtering making use of AI techniques and applying a gamemasters hints as to what is significant versus a Machine Learning adaptive behavior system.


The mechanism could be made to 'learn' from the gamemasters 'fixes', sets of adjustments that would be available for similar scenarios (added to 'suggested fix' lists for the human's manual adjustment. Factoring techniques applied to the scenario situation would be used in classification/summarization -- again with a human doing the defining/teaching/tweaking to create Judgement/asessment functions.



You will never be able to get the human teacher (gamemaster) out of the system as at some point you always have to instruct the system what results are good/bad and what factors are significant to the players experience (often very different depending on each scenario's theme/plot). The 'fix' solution types all have to be human defined/parameterized and kept within subjective bounds to fit a particular scenarios elements.

Such a system is much easier to create with a human at its center continuously adjusting/training it, than some much more automatic mechanism that would have to be pre-trained against a complex/varying game world domain .
A 'smart' tool ...



Another meta level tool is summarizing all the scenarios looking for player situations that are too similar (old complaint that quests are all mostly alike..) -- not just for finding scenarios that dont meet their designed 'player experience' specs/goal. The same metrics/collected data/classifications/assessments could be analyzed for similarities to point out changes that might be made to give more variations.
Again this is filtering/identification, but the filters/judgements could be based on 'training' by a gamemaster.

Machine learning is for solving problems that are difficult to solve manually. The NPC behaviors in many popular game genres are pretty simple:

- First Person Shooter: the NPCs run from point A to point B, take cover, and shoot things. Maybe in an advanced system NPCs have some awareness of their team and can perform some simple squad-based maneuvers.

- Platformer / Shooter: the enemies are basically set up for the player to kill them, and perform only simple patterns or flocking behavior.

No the NPCs are not realistic at all, but they are "good enough" to be fun for the player. Thus, with respect to those game genres the article linked in the OP is correct.

Unfortunately, there are also popular game genres where either: (1) players expect more realistic AI, or (b) the players want the AI to be hard to defeat.

- RPG: the point of these games (to part of the audience) is immersion in a realistic world. Recall Oblivion, which touted its Radiant AI system, that ended up being widely ridiculed:

http://www.somethingawful.com/d/news/asshole-physics-meet.php

Or check youtube for the thousands of videos about "radiant AI bug".

- MMOG: In todays industry standard MMOG, World of Warcraft, the "world" consists of static packs of 3-5 mobs that stand there until players kill them. Is it fun now? Yes. Would it be more fun with more realistic NPC behaviors? I think most players would say yes.

- RTS/Sports/Turn-based strategy: at higher levels of play there is a repeated complaint on every message board of every game in these genres "the AI isn't good enough without cheating". For the turn-based strat genre, really good AI is particularly essential because games like Galactic Civilization are just not feasible to play in a single online session.

Last DICE I talked to an EA exec and he said one of their most common complaints from the sports and RTS game enthusiasts is that the AI isn't good enough. Also, they are very interested to see if it is possible to make "fake players" for online games. That is, create AI players indistinguishable from real ones that participate in the online games to keep online numbers up. I assume it is to solve the chicken-egg problem of multiplayer games where creating a healthy online population requires an initial online population.

I'd venture to say that ML techniques could be worth it for these genres, because so far, due to their complexity, developers just cannot hand-code AI that is good enough for anything past casual-medium skill level players.