I thought Giraffe had achieved reasonable success with a NN based approach.

No, Giraffe's Elo on CCRL 40/4 is 2411, which is quite mediocre. There are many engines that are hundreds of Elo points stronger, including my own, RuyDos.

The problem is that many games seem much more complex than go, or don't require a killer AI.

This.

People need to remember that we aren't trying to "solve" something here... we are trying to create an experience. Often, that experience needs to include intentionally sub-optimal decisions and intentionally distinct characters. If you are trying to "solve" behavior, you get neither.

But actually, if we were to separate a civilization AI into several subproblems, like:

• which goal to pursue to reach victory conditions
• alliances and wars
• economy, research and production
• unit task/theater assignment
• unit micromanagement

Using ANN for diplomacy would be horrible, and economy lends itself much better to a search or utility approach, but wouldn't it make sense to use deep reinforcement learning for the unit micromanagement part?

It is the one that resembles zero sum board games the most, and its space state is quite huge, but not orders of magnitude larges than go (if we decide to limit it to the immediate surroundings of a unit and not the whole world map). It is also something regular game AI usually has trouble to handle.

I thought Giraffe had achieved reasonable success with a NN based approach.

No, Giraffe's Elo on CCRL 40/4 is 2411, which is quite mediocre. There are many engines that are hundreds of Elo points stronger, including my own, RuyDos.

Nice! Is there any place where you have documented your work on it?

It is the one that resembles zero sum board games the most, and its space state is quite huge, but not orders of magnitude larges than go (if we decide to limit it to the immediate surroundings of a unit and not the whole world map). It is also something regular game AI usually has trouble to handle.

When you consider multiple unit types and even multiple upgrades of units, it gets out of hand fairly quickly.

That is a fair point. But trimming the features down to a reasonable level is what we already do with utility functions. So what would be the problem with using ANN to fine tune the contributions of the utility function?

Some behaviours could be very hard to grade indeed, but some other not so much (like preventing the player characters from reaching their objective, or inflicting as much damage as possible before dying for a NPC).

If you have a fitness function that can tune an ANN to give you optimal play, why don't you just use the fitness function directly to evaluate candidate state changes?

If you have a fitness function that can tune an ANN to give you optimal play, why don't you just use the fitness function directly to evaluate candidate state changes?

I doubt he has that. Sometimes all you have is a database of previous games, where you have game situations you can feed into a neural network and have it try to predict the outcome of the game. You train an ANN to estimate the expected reward at the end of the game, which can then be used in a utility-maximization setup.

Indeed, I was only planning to score the final state for each character when he dies, but that is a good point. Even grading the utility of things accomplished by a given character through his lifetime could be hard to get right compared to only grading the final outcome for the whole team.

But I think it would still be "easier" than finding a good metric to grade the utility at each time.

But I think it would still be "easier" than finding a good metric to grade the utility at each time.

Unless you have actually practiced doing just that.

> People need to remember that we aren't trying to "solve" something here... we are trying to create an experience. 

I'm far more excited for applications beyond solving games. Neural networks can generate images, videos or sounds. This could make every NPC's voice unique (looking at Skyrim, with it's 5 voice actors), make realistic

" rel="external">animations (Spore creatures could evolve how they walk) or for example generate new biomes for Minecraft. Open world games could become much more diverse if neural networks can fill in the details. It's procedural generation that can actually work.

Are there any discussion groups for that?