Brief Introduction ------------------------- This is for a basic basic GDI+ graphical mud in which a player performs a secure login and is able to move around on a simple X,Y grid and is able to see other players on this simple X,Y grid. This is a client application sending packets via TCP/UDP to another application on a different computer acting as the server. Nothing is browser based. Here is how the logic goes so far. Client: Sends login/password information to Server application via TCP & SSLStream(.NET Framwork 2.0) (Secure). Server: Receives login/password via TCP & SSLStream(.NET Framwork 2.0). Server checks login/password against database. If user exists in database and password checks out do the following.... - Generate temporary GUID for user and put this into a hashtable in memory. - Add the users IP Address to a list of acceptable clients in which later the UDP listener logic will check this list to make sure we accept packets from people who have logged in properly. - Return the temporary GUID to the client. - Return an RSA encryption public key (changes everyday) to the client. Client: Receives the temporary GUID & RSA encryption public key from server via TCP & SSLStream(.NET Framwork 2.0). Because of these events I declare the user logged in. Client: Player now has the option to move around on the X,Y graphical grid. So the user presses W to move forward. On key down client launches an encrypted packet (using the RSA public key encryption) via UDP to server. The packet consists of the players temporary GUID, the type of packet(movement), and the direction the player wants to move. Server: Receives packet via UDP and decrypts the packet using RSA private key. Server checks to see what type of packet this is, since it is movement the server starts to move player of GUID forward UNTIL the client lets go of their W key in which on key up the client will send a request to stop. Here is where I'm stuck and need your help friends! What is the best logical way to send back player movement data that has been processed on the server? In a pulse? Like every second I should be updating all players of what is going on in their immediate area? I check the players X, Y radius and if other players exist in their area send back that data? What if they are alone? Then I just send them back their updated movement data? When should I graphically and visually update a clients position on the graphical X,Y grid? Only when I get a response back from the server for sync right? Do you see where I'm getting at? How do I handle the logic in this department? Does UDP have a feature where I give it a list of IP Address's to broadcast back to? Should I just make a loop and iterate through all these ip address's sending back player data? For example: If 800 players are online should i be looping through 800 ip address's sending player position packets every second? If I do this should I use only one UDP Socket which loops with a thread.sleep(10) milliseconds? Is that fast enough? I've tried sending UDP packets without using sleep and it sends them so fast I lost a lot of the packets. Should I open one UDP Broadcast thread for every 100 players and pulse to those players every second? Whew..... LuzArius "Can ya spare some cutter me brother?"...

Ugh, first off, encrypting movement individually is pointless.
- If you don't check for valid moves and movement speed on server, you've gained nothing
- A server updating 800 clients using RSA encryption will be big (as in expensive). Even if money is not a problem, you'll need to solve scalability issues of using distributed system. And a huge ammount of CPU will be busy just doing the encryption.

Calculate maximum movement a player can make in a given time step. Each time you receive an update, check last and new position. If it exceeds your maximum distance, the client is speed/teleport hacking. It is impossible in game mechanics to travel a larger distance in given time. It doesn't eliminate possibility of local movement hacks withing this distance, but this will be limited by short updates (warping every 250-500ms wouldn't acomplish much).

Quote:

What is the best logical way to send back player movement data that has been processed on the server? In a pulse? Like every second I should be updating all players of what is going on in their immediate area? I check the players X, Y radius and if other players exist in their area send back that data? What if they are alone? Then I just send them back their updated movement data?

Yes. Make some proximity check. If there's nobody in the area, you will end up with no updates. No problem here.

You generally do not check if something has been processed on server. You would want to have two "threads".

One that updates your character in your time.
Second, that parses the messages from server.

First one makes periodic checks to make sure player sees the character where server thinks the player is. You usually make these checks at 2-5x times the network latency. If client is showing player very close to where server thinks it is, do nothing. Otherwise, warp the character to last confirmed position. In case of lag/disconnect, the player will simply rubberband back (bad, but nothing else you can do).

Second one parses messages from server. Each time movement update is received, store the server's last update and time of update for use in first thread. That's it.

Quote:

When should I graphically and visually update a clients position on the graphical X,Y grid? Only when I get a response back from the server for sync right?

Horrible, you'd have lag of 50-500ms. You update player's character locally, and just keep it in sync periodically as stated above. You update other players when you receive their new position, and interpolate movement if needed.

Quote:

Does UDP have a feature where I give it a list of IP Address's to broadcast back to? Should I just make a loop and iterate through all these ip address's sending back player data?

Yes, UDP broadcast. But that only sends "same" message to all. You will want to minize traffic, so you'll iterate through clients manually, and send individual updates to each one.

Thanks for taking the time to respond.

Quote:

Ugh, first off, encrypting movement individually is pointless.

So when the client sends a movement packet to the server via UDP the packet should be completely uncrypted? Meaning the users GUID identifier (which identifies the user) should be unencrypted as well as the request to move? What if I place a sequencing system in the packet as well so at the server level I can weed out extra illegal packets sent (speed hack or something). Are you sure the movement packets should be left uncrypted?

Quote:

- If you don't check for valid moves and movement speed on server, you've gained nothing

What is the best way to check for movement speed on the server? Should the client send a sequence order in the movement packet so I know which packets received are legit?

Quote:

A server updating 800 clients using RSA encryption will be big (as in expensive). Even if money is not a problem, you'll need to solve scalability issues of using distributed system. And a huge ammount of CPU will be busy just doing the encryption.

How about TripleDES instead of RSA? Can you recommend a good encryption that a lot of the big boys use for their mmorpgs? Of course I'm not shooting for a big mmo but I'd like to have some strong security. RSA sounded great but I guess it is too much overhead like you said.

Quote:

Calculate maximum movement a player can make in a given time step.

Time steps. So if the user hits W one time it will move him one square forward. I should calculate how much time that takes? Then ensure that normal movement always fits within that timeframe?

Quote:

Each time you receive an update, check last and new position. If it exceeds your maximum distance, the client is speed/teleport hacking.

When you say check last and new position you are talking about the players position in memory server side right?

Quote:

It is impossible in game mechanics to travel a larger distance in given time. It doesn't eliminate possibility of local movement hacks withing this distance, but this will be limited by short updates (warping every 250-500ms wouldn't acomplish much).

aren't local movement hacks simply cosmetic to the client but the real values are on the server?

Quote:

Yes. Make some proximity check. If there's nobody in the area, you will end up with no updates. No problem here.

Hmmm so there are actually two things going on. If no one is in the area then I will just update that single user's position?

Quote:

You generally do not check if something has been processed on server. You would want to have two "threads". -One that updates your character in your time. -Second, that parses the messages from server.

So whatever the user does client side let it happen visually. But then use the messages received from the server to make sure the client is indeed in somewhat sync. What if I'm making this tile based mud very very twitch combat orientated where a response time means everything? Will this concept still work?

Quote:

First one makes periodic checks to make sure player sees the character where server thinks the player is. You usually make these checks at 2-5x times the network latency.

Hmmmm so I have to actually calculate the latency that exists between client and server then do a calculation based upon that?

Quote:

If client is showing player very close to where server thinks it is, do nothing.

What if the character position is off by one tile visually on the client? I was hoping by making the game tile based I could achieve some sort of precise sync.

Quote:

Otherwise, warp the character to last confirmed position. In case of lag/disconnect, the player will simply rubberband back (bad, but nothing else you can do).

This I understand.

Quote:

Second one parses messages from server. Each time movement update is received, store the server's last update and time of update for use in first thread. That's it.

cool :)

Quote:

Horrible, you'd have lag of 50-500ms. You update player's character locally, and just keep it in sync periodically as stated above. You update other players when you receive their new position, and interpolate movement if needed.

Once again even if this is tile based movement?

Quote:

Yes, UDP broadcast. But that only sends "same" message to all. You will want to minize traffic, so you'll iterate through clients manually, and send individual updates to each one.

If I interate through clients manually via UDP and lets say 800 players are online. Do I just loop through them real fast or should I add a thread.sleep(10) in between so each packet has time to go through?

Hey thanks for all your help, this has been sooo nice. Whoever reads this thread in the future will also be greatly educated because of your input. THanks man!

Luzarius

Quote:

Original post by luzarius Client: Player now has the option to move around on the X,Y graphical grid. So the user presses W to move forward. On key down client launches an encrypted packet (using the RSA public key encryption) via UDP to server. The packet consists of the players temporary GUID, the type of packet(movement), and the direction the player wants to move.

I'm not sure what you mean by this exactly. If you mean what I think you mean, then you probably should not send a GUID identifying what user it comes from in the packet. That should all be tracked by the server. If you're using something in the packet to identify the user (where the packet is coming from), anybody could send a false packet (say to send a message to somebody... or like spoofing).

Even with encryption, it's still possible.

You could track clients by a GUID, but I think it would be best just to keep that info on the server.

Quote:

Quote: Ugh, first off, encrypting movement individually is pointless. So when the client sends a movement packet to the server via UDP the packet should be completely uncrypted? Meaning the users GUID identifier (which identifies the user) should be unencrypted as well as the request to move? What if I place a sequencing system in the packet as well so at the server level I can weed out extra illegal packets sent (speed hack or something). Are you sure the movement packets should be left uncrypted? Quote: - If you don't check for valid moves and movement speed on server, you've gained nothing What is the best way to check for movement speed on the server? Should the client send a sequence order in the movement packet so I know which packets received are legit?

Compare players last destination with new location. Compare this with maximum possible step in that time frame. If it's larger, you get position hack. Client can then do anything they want, if they apear at wrong position, they hacked. This of course depends on actual world definition, movement rules, and similar, but stay independant from protocol.

Quote:

Quote: A server updating 800 clients using RSA encryption will be big (as in expensive). Even if money is not a problem, you'll need to solve scalability issues of using distributed system. And a huge ammount of CPU will be busy just doing the encryption. How about TripleDES instead of RSA? Can you recommend a good encryption that a lot of the big boys use for their mmorpgs? Of course I'm not shooting for a big mmo but I'd like to have some strong security. RSA sounded great but I guess it is too much overhead like you said.

One MMO familiy is using RC based block cipher (possibly modifier RC4, simple challenge based handshake. Keys can also be switched at any time, to prevent playback attacks. The encryption here is rather weak, since starting sequence results in easily crackable code, but it is sufficient to deter basic hacking.

One of more recent CRPGs is using TCP based stream cypher combined with stream compression.

Don't worry too much about security, just apply something that can be calculated fast (several instruction per byte maximum)

Quote:

Quote: Calculate maximum movement a player can make in a given time step. Time steps. So if the user hits W one time it will move him one square forward. I should calculate how much time that takes? Then ensure that normal movement always fits within that timeframe? Quote: Each time you receive an update, check last and new position. If it exceeds your maximum distance, the client is speed/teleport hacking. When you say check last and new position you are talking about the players position in memory server side right?

Yes.

Quote:

Quote: It is impossible in game mechanics to travel a larger distance in given time. It doesn't eliminate possibility of local movement hacks withing this distance, but this will be limited by short updates (warping every 250-500ms wouldn't acomplish much). aren't local movement hacks simply cosmetic to the client but the real values are on the server?

If you do not check them, or put too much trust on client, the client-side error can propagate to the server. So always check movement on the server.

Quote:

Quote: Yes. Make some proximity check. If there's nobody in the area, you will end up with no updates. No problem here. Hmmm so there are actually two things going on. If no one is in the area then I will just update that single user's position?

Yes. This is to ensure that after a while, client does not get out of sync with server (Due to animation, approximation, rounding, latency, lost packets, etc.)

Quote:

Quote: You generally do not check if something has been processed on server. You would want to have two "threads". -One that updates your character in your time. -Second, that parses the messages from server. So whatever the user does client side let it happen visually. But then use the messages received from the server to make sure the client is indeed in somewhat sync. What if I'm making this tile based mud very very twitch combat orientated where a response time means everything? Will this concept still work?

Yes. To ensure smooth interaction, immediately act locally, but periodically, enforce server's authoritative state, if needed.

Quote:

Quote: First one makes periodic checks to make sure player sees the character where server thinks the player is. You usually make these checks at 2-5x times the network latency. Hmmmm so I have to actually calculate the latency that exists between client and server then do a calculation based upon that?

Keep a time stamp. This is to avoid too frequent warping or rubberbanding, even if it doesn't affect gameplay.

Quote:

Quote: If client is showing player very close to where server thinks it is, do nothing. What if the character position is off by one tile visually on the client? I was hoping by making the game tile based I could achieve some sort of precise sync.

If your character can ever only stand in one specific tile, then you do not need to deal with approximation. That is only needed for smooth movement.

Quote:

Quote: Horrible, you'd have lag of 50-500ms. You update player's character locally, and just keep it in sync periodically as stated above. You update other players when you receive their new position, and interpolate movement if needed. Once again even if this is tile based movement?

Yes. Movement would still suffer. Effects would be the same, just on per-tile basis.

Quote:

Quote: Yes, UDP broadcast. But that only sends "same" message to all. You will want to minize traffic, so you'll iterate through clients manually, and send individual updates to each one. If I interate through clients manually via UDP and lets say 800 players are online. Do I just loop through them real fast or should I add a thread.sleep(10) in between so each packet has time to go through?

That depends on platform, network code, system, network itself.

You could try using non-blocking sockets, and after you're done processing one player, send their data to socket, and let network layer or the OS deal with it.
You could batch everything for all players, then after you finish with update loop, just send the packets.
You could put packets to be sent in a FIFO queue, then let one or more threads take care of them.

Unfortunately, when you start talking about big numbers, system limitations often affect the design. Start with simplest naive solution, then improve if that becomes an issue.

I know that in case of bulk updates (1000+ clients on single machine) the threading implementation becomes problem, so a hybrid (2-10 threads with queue) solution proves most efficient (Linux, with special kernel build).
Typical MMOs scale to 200-300 users per server zone. This is also due to limitations of actual action processing, database access and world updates.

Note that all of the above is based on my perspective, and there's always room for error. What I propose is not guaranteed to be best. It's just how I personally would tackle the problem. And probably change a few things as I'd go along.

Quote:

Original post by xrazybud Quote: Original post by luzarius Client: Player now has the option to move around on the X,Y graphical grid. So the user presses W to move forward. On key down client launches an encrypted packet (using the RSA public key encryption) via UDP to server. The packet consists of the players temporary GUID, the type of packet(movement), and the direction the player wants to move. I'm not sure what you mean by this exactly. If you mean what I think you mean, then you probably should not send a GUID identifying what user it comes from in the packet. That should all be tracked by the server. If you're using something in the packet to identify the user (where the packet is coming from), anybody could send a false packet (say to send a message to somebody... or like spoofing). Even with encryption, it's still possible. You could track clients by a GUID, but I think it would be best just to keep that info on the server.

Ahh i see. So how about this.

Server: receives packet. server identifies that this is a movement packet. Server finds out that this packet came from IP Address 63.4.3.1 and since the user logged in earlier we know that the person at this IP address is player X.

Is that a good way for the server to identify a user when the user tells the client to send a movement packet to the server? The IP Address? maybe some of their host or dns information?

Also once again should any of this be encrypted?

Antheus thanks that helped clear a lot up.

I have access to Rijndael encryption/decryption methods. I heard it is fast and efficient. Or as you said I could just use rc2 or something.

[Edited by - luzarius on April 20, 2006 8:36:49 PM]

For encryption, assuming you want symmetric cyphers, I would use X-TEA (as found in the FAQ). It's very fast, and seems to be very secure even with just 8 rounds (they recommend 32 rounds). It also seems to be patent free.

Quote:

Original post by luzarius Ahh i see. So how about this. Server: receives packet. server identifies that this is a movement packet. Server finds out that this packet came from IP Address 63.4.3.1 and since the user logged in earlier we know that the person at this IP address is player X. Is that a good way for the server to identify a user when the user tells the client to send a movement packet to the server? The IP Address? maybe some of their host or dns information? Also once again should any of this be encrypted?

struct client {
int fd;
char buffer[buffer_len+1];
char name[name_len+1];
}

Just as an example, if you read information from client.fd, and get a packet out of client.buffer, you can use the client structure to get the name for that player. You could still use a GUID to identify the player instead of the name.

If the server is also checking to see if everything is valid, I think encryption on movement packets would be more of a preference. Like if the protocol was proprietary, you didn't want anybody making custom clients, stuff like that.

TEA (and X-TEA) are patent free. They're also VERY fast, although blowfish / twofish can be adapted also for speed and are also patent free. However because blowfish/twofish have a large initialisation state change based on key (expensive), you should to carry the entire state around for speed if you're using more than one key, and take the hit in RAM rather than massive latency.