There's an unrealized issue here: "Perfect" AI is not fun. Play Quake3 with Xero on nightmare difficulty on the last level. He's pretty much a near perfect aim bot with a rail gun who will one-shot you just about every time. It's only possible to beat him because he's not "perfect". If he was perfect, he'd be impossible to beat.
You're right. It's because optimizing the AI to be as good as possible at the game is not fun for players, because today most AI can be made to be much stronger than any human player (in FPS games, chess, strategy games, etc.).
Instead of having the AI optimize for the goal of "play the game as strong as possible", what if its goal was "play the game in such a way, that the human player playing it has the most fun."
That's much harder, yes. The AI algorithms would need to make use of human psychology knowledge and such, but the end result might be better.
One example where I remember a weak point of AI was when playing one of Assassin's Creed games. Near water, the AI was chasing me, and many of them ended up jumping into water and dying. It was very sad. If the AI had predicted the results of its actions, it would've seen that "jumping forward" would result in its death, and it probably shouldn't do that.