Operant Conditioning

GameDev.net · 2010-09-26T06:14:37

Hi allI have a school project coming up for which I'll have to write an XNA game in C#. I was thinking of an evolutionary game in which you set the goals, and let things evolve as the game engine selects the best specimens to survive. What I mean by specimens are cells which are basically circles which can move, have some properties, sensors and weaponry.I'd like some advice on the AI I am thinking of using for the cells.I'd like my cells to gradually learn through trial and error, basically through operant conditioning (i.e. rewarded behaviour will be more likely to return when the same (or similar) stimuli are observed again, the converse is true for punished behaviour). I already have an idea of what behaviour should be rewarded, and what should not (they are born with a particular set of events to reward mappings). To get into more detail: * Using its sensors the cell will collect events/stimuli.* Events are looked up in the reward table, and rewards are given to responses given in last x iterations/seconds.* It will then look in its dictionary of events to responses mappings. Responses looked up like this will have their activation value increased or decreased, depending on whether this behaviour was rewarded or punished before.* After this lookup, all responses with an activation value exceeding a certain treshold will then be activated on the cell (e.g. it would fire gun 1).My concerns/questions are:* Would this approach work? Would it take way too many iterations for them to learn? (I'm not looking for an incredibly clever AI; as long as they learn using something akin to operant conditioning, and don't just run in circles, I'm happy)* How can I keep this performant?* How do I limit memory usage to something reasonable as they learn and store more and more events to responses mappings with a reward value? Or would this not be a big problem after all?* Is there an existing AI library that may ease development for this AI?* Are there any documents, ... about this? (I tried a couple of searches but my search terms seem poisoned by many other results)* Any ideas on how to handle continuous input/output values such as distance, and speed. (I might divide them up in y zones; not having to do so would be better though, both for memory as for accuracy)Any help/comments are very much appreciated.

Artificial Intelligence Programming

Started by GameDev.net September 17, 2010 08:50 AM

1 comment, last by GameDev.net 14 years, 4 months ago

IADaveMark

3,839

September 17, 2010 09:10 AM

A) Look up "reinforcement learning" as it applies to game AI. (The game "Black & White," whose AI was done primarily by Richard Evans, was a famous and successful implementation of this.

B) Start small and iterate. Don't plan yourself over your head. A smaller, completed project that works is better than a list of nifty ideas that you couldn't pull off.

Dave Mark - President and Lead Designer of Intrinsic Algorithm LLC
Professional consultant on game AI, mathematical modeling, simulation modeling
Co-founder and 10 year advisor of the GDC AI Summit
Author of the book, Behavioral Mathematics for Game AI
Blogs I write:
IA News - What's happening at IA | IA on AI - AI news and notes | Post-Play'em - Observations on AI of games I play

"Reducing the world to mathematical equations!"

Operant Conditioning

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Operant Conditioning

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines