In terms of Winsock,

if the client sometimes needs to send 100 bytes and sometimes 1000 bytes for example, how is the server supposed to know before-hand the buffer size for the incoming send() from the client of 100 or 1000 bytes that i would need in recv() on the server?

Should i just set them both to a limit that i know for certain it won't reach, say 2048:

Client:
send(socket, cstr, 2048);
 
Server:
recv(socket, recvbuf, 2048);

Or should i do like the Winsock example and save packet space by when i send() by capping the buffer size at the c-string's size:

Client:
send(socket, cstr, strlen(cstr));
 
Server:
recv(socket, recvbuf, 2048);

Questions:

How do i determine the buffer size to use?

Should the send()'s buffer size always be capped at the length of data im sending(strlen(ctr)), or should it just be 2048 for example across the whole program?

Am i wasteing 2000+ bytes worth of packets if i do something like this

send(socket, "Username=Jimmy".c_str(), 2048);//not proper syntax i know

?

You can deal with this in a few ways. The two most obvious methods are:

• Start each message with a number that indicates how many more bytes you're sending (usually you would send this in binary in network byte order but you could use text as well - you'd just need to terminate the number's text in that case. HTTP sends content-length as plain text for example).
• End each message with a special terminator sequence (like how C strings have null terminators).

I usually prefer the first method because it's easier to write a receive loop if you know the expected number of bytes ahead of time.

In the first method, your receive code will look somewhat like this:

• 'select' to find out if there's any data available to read. (Optional - if your code has nothing else to do you can skip this step and go straight to reading)
• 'read' the number to find out the size of the next message (if you use binary you can use a fixed number of bytes such as 2 or 4. If you use text, you have to read until you reach the terminator character).
• 'read' in a loop until you receive that many bytes.

Shawn, the last project I worked on used Nypyren's first suggestion and it is by far the eaiest way to manage this problem. Just send the size first.

My server did it in this way, packet:

byte - packet id

signed short (2 bytes) - packet size

body (N bytes) - packet body

First alloc any size you want, put there required information, when you done, set packet id, calculate size of output buffer and set it to 2,3d bytes.

If your packet has constant size, and has id = 1, for example:

id(byte)

size(sshort)

account(char 16)

password(char 16)

save_password(byte(bool))

total 36 bytes, there are always 36 bytes, and you know that client have to read 36 bytes for packet with id 1.

If you have dynamic size packet with id 2:

id(byte)

size(sshort)

message(string)

You have always 3 bytes in begin and some bytes in the end. You make dynamic buffer, put there your string, for example("Hello, there!") - 13 bytes. And set it to packet.

id: 2

size: 16

message: "Hello, there!"

You read first 3 bytes to get id, and size, and then read whole string.

But i guess dynamic strings you can set only to last value. If you have string in middle of the packet, you should know about that and set constant size for this string.

The way I have solved this is the following:

Set both the send and recv buffers to some pre-defined max-size, for me that is 2048 bytes or 2kb. No packet sent can be larger than this, ever. This allows you to pre-allocate as many 2kb buffers are you need to handle all incoming packets, and re-cycle them by just zeroing the memory again, without the need to free/alloc new memory every time. My game is also written in C#, so this lessens the pressure on the GC a lot, allowing me to recycle packet buffers and not having to re-size them, ever.

So, about knowing how much "data" you can read from each received packet, most (all?) socket layers return the amount of bytes read from the recvfrom call. But since we usually deal with bits and not bytes, this only gives us an upper bound on the amount of bits (received bytes * 8) that we have. There are three different ways to deal with getting the exact data out of the packet that I have found, as an example I am going to say that I need to send 5 RPC calls and 2 Player Data objects

1. The most common and obvious one is to have the first two bytes of the packet be the exact size, in bits, of the packet, so the packet would look like this: [size][rpc][rpc][rpc][rpc][rpc][playerData][playerData]
2. Know what data you sent in the packet, and pre-fix each block of data with the count of that type of data, for example if i need to send five RPC calls and eight player data objects, my packet would look like this: [5][rpc][rpc][rpc][rpc][rpc][2][playerData][playerData]
3. Before each segment of data, write one bit set to 1. Before you read each piece of data, you read this bit, if it's 1 you should read the data, if it's 0 you are done, so the packet would look like this: [1][rpc][1][rpc][1][rpc][1][rpc][1][rpc][1][playerData][1][playerData]

Personally i use the third approach because the other two methods require you to "go back" and write the size/count after you have serialized all the data, and it creates an awkward logic flow. The third approach allows each write to be completely self-contained with no need to refer to anything outside of itself.

The question is concerning to me, because it seems like you think that a single call to send() will end up in a single call to recv() on the other end.

This is not the case.

If the sender sends one message of 15 bytes, one message of 238 bytes, and one message of 1 byte, the recipient may receive:

- one message of 15 bytes, and one message of 239 bytes

- or one message of 2 bytes, one message of 4 bytes, one message of 8 bytes, one message of 16 bytes, one message of 32 bytes, one message of 64 bytes, and one message of 128 bytes

- or one message of 254 bytes

- or some other sequence

This assumes there is no failure in the network, in which case the recipient may receive nothing.

The receiver in a TCP based system should pretty much always keep a fixed-size receive buffer, and each time calling recv(), call recv() for as much as is left in that buffer. If you use a cyclic buffer, it means you don't have to copy/move data as you dequeue it, but you have to deal with the "break" at the end.

Once you have received data into the buffer, you can start looking at it to see if you have received a full message from the other end (whatever that means,) and if so, process and remove it. Repeat.

The question is concerning to me, because it seems like you think that a single call to send() will end up in a single call to recv() on the other end.

This is not the case.

If the sender sends one message of 15 bytes, one message of 238 bytes, and one message of 1 byte, the recipient may receive:

- one message of 15 bytes, and one message of 239 bytes

- or one message of 2 bytes, one message of 4 bytes, one message of 8 bytes, one message of 16 bytes, one message of 32 bytes, one message of 64 bytes, and one message of 128 bytes

- or one message of 254 bytes

- or some other sequence

This assumes there is no failure in the network, in which case the recipient may receive nothing.

I don't know if you're referring to Nagle's algorithm, but i've never had a situation where a remote client would recv() partial bytes, like 2 bytes of 100 bytes for example. Either the client receives the server's send() message in full or a winsock error message (ie: error code 10035) is created, per the return value of recv(). Or maybe you're referring to a client that has a network connection that's cutting in and out

The receiver in a TCP based system should pretty much always keep a fixed-size receive buffer, and each time calling recv(), call recv() for as much as is left in that buffer. If you use a cyclic buffer, it means you don't have to copy/move data as you dequeue it, but you have to deal with the "break" at the end.

Once you have received data into the buffer, you can start looking at it to see if you have received a full message from the other end (whatever that means,) and if so, process and remove it. Repeat.

Good, this is exactly what i'm doing and it's working perfect.

I don't know if you're referring to Nagle's algorithm, but i've never had a situation where a remote client would recv() partial bytes, like 2 bytes of 100 bytes for example. Either the client receives the server's send() message in full or a winsock error message (ie: error code 10035) is created, per the return value of recv(). Or maybe you're referring to a client that has a network connection that's cutting in and out

Maybe it's rare, but it does happen and it is a case that needs to be handled.

I don't know if you're referring to Nagle's algorithm, but i've never had a situation where a remote client would recv() partial bytes, like 2 bytes of 100 bytes for example. Either the client receives the server's send() message in full or a winsock error message (ie: error code 10035) is created, per the return value of recv(). Or maybe you're referring to a client that has a network connection that's cutting in and out

It's easy to have this happen if you encounter packet fragmentation, e.g. sending a packet larger than routable MTU. It isn't exactly rare if you're sending nontrivial amounts of data.

My guess is you simply haven't been paying attention. This is not particularly rare -- over the greater internet, with your own hosting and a remote user, you will possibly see it on up to a percent of all packets.

I am not referring to the bad case of networks that are physically unreliable. A simple packet collision on any segment between sender and receiver is enough to make this happen.