Greetings, mighty developers!

with the company i'm working for we are building a framework that helps employ Reinforcement Learning solutions inside their games, where "Reinforcement Learning" is a fancy word meaning "let your NPCs learn to behave automagically". Our goal is to reduce the gap between the theory and the production needed to make intelligent, more challenging and realistic NPCs.

So, if you're interested in exploring the possibilities of reinforcement learning for game AI, please don't hesitate to reach out or add your comments.

Our question is: would you use such a RL solution? and for what? And if you are not interested, why not?

It's been done before, both good and bad.

Machine learning is great. It's useful for a lot of things. It can be used as a component in training up complex behavior. There are drawback like it doesn't allow for designers to tune, it is difficult/impossible to understand the relationships the algorithm forms, and so on, but for many things like gesture recognition, voice recognition, and figuring out what a player is doing to reduce it down to a small set of possibilities, they work great. You can use machine learning to train up many different things in games.

Machine learning to train up, then lock the training when you ship? Go for it. Machine learning to train about gestures, and fine-tune to better read what the player is doing? Sure. I've used it on several game titles. I've used it to help identify Wii remote motion, and I've used it in VR to detect hand gestures, until better third-party libraries were developed. It has a lot of great uses.

But there is a critical problem when applying it for “more challenging” play: Machine learning doesn't train to produce fun.

When used against a player, machine learning will adapt to whatever the human is doing, and eventually overcome it. The more you play, the more the computer defeats you. No matter how good the human is, eventually the machine will ALWAYS defeat them, the human will ALWAYS lose. That isn't fun.

Totally agree with you, but the idea is to learn for example NPCs that play following a behaviour which is defined by the game developers, where the NPCs learn through the direct interaction with the environment, actually playing it and choosing actions given what they see about their world. Do you think that something like that can be useful?

I see no value in it. What am I missing, what does it gain you over simpler behavior systems? NPCs can be built as complex as The Sims if you wanted, built with systems more typical of games that human designers have complete control over. The design would abandon that control, but I don't see what is gained in exchange.

I hope this video is relevant. It's very interesting regardless.

@frob the missing part is that maybe sometimes a more complex behaviour is needed: what happens there? Nowadays, ChatGPT is an example, complex behaviour can be learned by human preferences, where, in this particular case, the game designer decides how the agent should behave by just deciding visually the best behaviour between multiple offered

There is one way to use Machine Learning in game design that could be interesting: Train an agent to play your game extremely well and see if it makes use of all the aspects of the game. For instance, if you have six weapons available but the agent uses only one or two, there's probably an opportunity to improve the game design to create circumstances where other weapons have an advantage, or perhaps the stats on the weapons can be tweaked.