Hi. I am trying to make a poker bot that will play a decent game of poker. I've just finished working on a program that use combinatorial mathematics to evaluate a flop and hand and come up with a decision but I feel that I have failed at that since it basically sucks. So I'm back looking at some other ideas perhaps rule based. For instance, if you have this and the flop is this do this unless your opponent does this. I'm not sure how to program something like that. I've read some things on a minimax algorithm but don't know much about that. Understand that I'm not trying to make it world class. If I can just program a bot to play the way I play whether that be dumb or smart then I will feel I succeeded if it isn't that great. I have started on a datafile in which my program will open this file and find a code and decide what to do based on that code but that seems to be quite an undertaking if you consider all of the possibilities. Oh yeah, the game is Texas hold em. Anyway, I have the preflop decisions for the program done and that isn't a problem but the problem is when the flop hits. Sure, I could spend the next month entering mindless codes into a textfile but is there an easier way. I've already looked at the bayesian poker thesis and Alberta research and the Poki bot but that stuff seems over my head and remember, I'm not trying to make it beat Poki, I just want it to play the way I do. Any ideas, or should I go back to entering mindless codes until I lose my mind.

From a purely stagnant point of view, you could just program a straight poker player - bets with top pair, reraises with two pair, etc. etc. Much of that logic should be pretty easy to beat by any experienced player, in that it is quite translucent and stale.

Hard coding certain situations might not be a bad idea, as every poker player, good or bad, has his/her own rules and tendencies of play. A neat idea would be to give your bot some attributes, say, strength% (weak::tight), or discipline% (loose::aggressive).

But the absolute hardest part above all I think would be to enable the bot to actually play against other people (or bots). To be able to play against others (rather than just playing the cards) takes a certain AI concentrating on what the bot thinks the other player has. Like I said before, examining the board alone for decisions takes much of the guesswork out of playing the bot, ie., he would probably lose much of the time against human players.

For example, if the flop comes AK9 (rainbow), and your bot bets with KJ, the situation gets pretty monotonous, especially if you tell your bot to always fold if reraised here. So, a good player with Q9 will always win this battle.

To sum, I think it would be counter-productive, if not impossible to design the AI without calculating such things as other players' tendencies, weaknesses, past plays, etc. into the mix. Without those, the bot has no chance.

Quote: Original post by theredpill99
Hi. I am trying to make a poker bot
...
For instance, if you have this and the flop is this do this unless your opponent does this. I'm not sure how to program something like that.

The syntax should be easy enough:

OnPlayerXBet()if( bot->hand > playerx->hand )   bot->raise;else   if( bot->can_draw_a_better_hand )      bot->call;   else      bot->fold;

The difficulty arises in deciding whether or not the bot thinks he's beat. Throughout a game, the bot should build a database of playerx's moves, and calculate a % of what the player might be holding and put it into playerx->hand. Of course this % will change after the flop, turn and river cards, ideally reaching close to 100% (the bot knows for sure what the other player has). I'll think about this some more and get back to you.

All the information concerning the best online poker player can be found here:

http://games.cs.ualberta.ca/webgames/poker/

They even have servers with which to test you bot.

The best approach I can see is to assign influence values to hands, and actions.

So for example, your starting cards get a certain value, let's say from 0-1, and any hand with a current value of say <0.5 gets folded. Obviously 72 would have a very low starting value and be folded unless it's in the big blind (in which case you'd add 0.5 to every hand).

Now if someone raises preflop, that would have a negative influence to the value of your current hand. The size of the raise would scale the value. So if you get K9s and the base value for that hand indicates to limp in, say 0.5-0.65 in value, and then someone raises ahead or behind, that would have a minimum set effect, say -0.15, that would also be scaled if the bet was beyond some threshold, say 2-3x the big blind.

So in this case if you get a raise ahead the bot may decide to play it but if it gets a raise behind, the value of the hand falls below the fold threshold because you can assume that you're hand is dominated.

Post flop, it's more of the same, you have fixed assigned base values for made hands, with various modifiers. For example, bottom pair is not equivilent to top pair with a good kicker. Further, top pair with a good kicker is not the same in a heads up game compared to a game with 5 or more people still in after the flop.

So you'd have values for things like bottom pair, middle pair, top pair, two pair, three of a kind (seperate values probably best for a set or trips), and so forth then modifiers for things like kicker, players in the hand and so forth.

You calculate all of that after the flop, and then again based on others actions you modify the hand value.

You could change the behaivor of the bot then by altering the values at which it does different things. So for example a looser version of the bot, might sometimes call things down to 0.2 or lower and so forth.

The final piece of course would be to track what other players do and use that in the equation. Are they very tight, as in do they only bet when they have a high value hand? You can keep fairly basic stats on other players, where they fold, how they bet (at what stage in the game these actions occur) and the average value of their hands you've seen. So over the course of a few hours of play the bot will see hands the other players have played and can easily calculate it's own internal value it would assign to that hand. Then keeping an average or history of that it can look back and consider that player x may have played through the river with relatively weak hands before, calling any bets (calling station pattern), and act accordingly.

cool, sounds like you play poker, yourself. I'm not sure I have the time to do it. I believe I can do it but I'm burnt out right now. I'm a beginning programmer and it took me weeks to get the evaluate hand percentage down. I mean it is a program that calculate your percentage according to certain flops and hands. It also reads the cards off the screen and also knows what position you are in the hand automatically. What it doesn't do yet is know the pot size but what pisses me off is that I could have used a way to just capture the running text and it would have been easier than reading the graphics. It also does the turn card, too but that is still a ways off from having a bot that earns money. If I finish it, I'll let you know but I'm burnt out right now.

Dislamer:i do not know very much about pker, what i do know, i got off This thread

First start with a table of all combinartions of 5 cards (IIRC 52*51*50*49*48 = 311875200 Different possibilities).

you then figure out which cards each other player could have, and then figure out the probability of each player having them.

you then develop a profile of the player, things like what the player does when the player has a comobination of cards, or a suite or numbered cards, or a very bad hand, ect.

Then from the players behaviour, you then retune the probabilities of each player having each combination of cards. From that, you find every combination of cards which beats yours (and which other players could have) and from that find the probability that each player could beat you. and from that, the probability of winning/loosing.

Its from those sets of probabilities that you would then have a probabilistic rule set (so that your player won't try and do the same thing twice.

This is all very interesting... i'm probably going to look into this in more deatail soon.

From,
Nice coder

Quote: Original post by Anonymous Poster
First start with a table of all combinartions of 5 cards (IIRC 52*51*50*49*48 = 311875200 Different possibilities).

That's actually the number of permutations. See any of the following for clarification:

http://mathforum.org/dr.math/faq/faq.comb.perm.html
http://www.themathpage.com/aPreCalc/permutations-combinations.htm
http://www.math.cuhk.edu.hk/~gee243n/pdf/handout05.pdf
http://www.sunny.issaquah.wednet.edu/mathclub00/pdf/11lesson.pdf


-Will Dwinnell
http://will.dwinnell.com


[Edited by - Predictor on September 9, 2004 8:18:05 AM]

Quote: Original post by theredpill99
...It also reads the cards off the screen and also knows what position you are in the hand automatically...

...aw man, are YOU the guy who keeps taking 40 seconds to bet on partypoker?

Some general thoughts:

The fundamental theory of poker states that:

A) Whenever you do something different from what you should do if all hands were face up (perfect information), you are making a monetary mistake.

B) Whenever your opponent does something different from what he should do if all hands were face up (perfect information), he is making a monetary mistake.

From this you can derive that, in regards to bluffing, you should only bluff with the WORST hands you might hold in order to increase the chance your opponent will make a monetary mistake with his (he will fold many of his bad hands, which are better than the ones you will be bluffing with.. ie, hes making a monetary mistake)

Bluffing of course, should be done very selectively or else you run into making too many monetary mistakes yourself.

In regards to FUTURE money - ie, the value of a bet or raise in terms of winning EXTRA money not ALREADY in the pot:

The bot needs to think it has a BETTER than 50% chance to win the pot in order to make a bet or raise correct and that is because it might get raised/reraised. In essence when it bets $1 it is laying 2:1 odds if it will call a raise (It will lose $2 extra when it gets raised, and only win $1 (at best) extra when it doesnt) - This only holds true if it has a reasonable chance of winning by checking/calling (ie, bluffing is not an attractive option)

Before the last card the bot should be more liberal about betting hands in order to prevent its opponents from drawing for free - conversely it should be more conservative after the last card is out as players who have missed their draw will not be calling (the bet earns nothing extra) while the players that hit their draw will call or raise causes the bot to lose more money than necessary.

--

Temporal difference learning can be a strong tool in games like this if used right - for example you will want to measure its opponents tendency to call preflop but you need to guard against your opponent constantly changing gears making your estimation of his tendency to play inaccurate - so a learning rule something like:

OpPreflopTend = OpPreflopTend + 0.1 * (Played - OpPreflopTend)

Where Played is 1 if he played this time and 0 if he did not - this will keep a running average of how often he plays a hand with more weight given to his most recent hands. 0.1 is the learning rate and can be adjusted - higher puts more weight on the latest actions.

With a value such as OpPreflopTend you can then measure what hands your opponent might hold .. if he plays 20% of the hands he is dealt, you might begin with the assumtion that when he plays its probably one of the top 34 hands that you can hold preflop (20% of 169 preflop hands)


I have many more ideas but at the moment I must run..

- Rockoon