When I thought I was all done I tried to switch server/client machines (i.e ran the server on one computer, and the client on the other). Things didn't work out. I have have code that tries to merge data from several recv messages etc, but that fails? I'm using non-blocking, non-connected sockets (udp). Each udp packet contains a 32 bit size + crc header, it's sent via sendto (looping until all data has been sent). As hinted it works if I switch which computer runs what (client/server). It also works if I split the packages into smaller chunks using several sendto. It works if I randomly ommits to send any of the splitted chunks (the client just drops the faulty packet, when it assembled a valid packet). Anyone recognize this behavior? I've done a working client/server program alot of years ago using winsock/udp and I can't recall having this much trouble back then. Is this following true: sendto - when several calls are needed, sends pieces of the udp packet over internet? recvfrom - might or might not get these pieces? recvfrom - can get the pieces in any order? (How the heck can one assemble the pieces into the original message if they only recieve 2 unordered bytes at a time?). sendto - can buffer some piece of data before it send it, like: Some examples: I send "Hello world". Can sendto split the message into several pieces "Hello ", "world"? Can sendto add the message to some previous udp packet. "XXXHello world"? If no, is it enough to check for a new udp header at the beginning of each new recv block? Can the data recieved be "XX", "XHel", "lo wor", "ld"? Can the data recieved be "XX", "lo wor", "ld"? Can the data recieved be "XHel", "XX", "ld", "lo wor"? Can the data recieved be "XHel", "XX", "lo wor"? Thanks in advance.. Sorry for the bad language and formulation, it's a combination of english not being my native language and working 16 hours a day for the last week :( I hate deadlines.. especiality the really important ones... [Edited by - eq on May 3, 2005 2:34:50 PM]

The UDP header is not visible to you at the socket layer.

UDP will not split a packet. With UDP, a single call to send() results in a single delivery for recv() in general. The packet may be dropped (zero calls to recv()) or duplicated (multiple calls to recv() return the exact same data), but packets are not sliced or coalesced as with TCP.

UDP (and IP) may fragment the packet if it's sent through a channel with too small MTU, but that's invisible to you, the user of UDP.

If you send "123 Hello, world", you will see zero or more packets containing exactly "123 Hello, world!". You won't see "123 " or ", world!".

For your question about getting things in order, how do you make sure you do that.

Place a time stamp or a sequence number in the packet you are sending. A sequence number would probably be best and take up less space. Depending on how many message you are going to send, it could probably even be a 16-bit or even 8-bit value. Make it unsigned and you'll be able to handle 256 sequence numbers with 8-bits, which may or may not be enough for you. Once you get to 255 restart at zero.

For your question about dropped packets.

Sequence numbers will also help, if you get through a window of N and you haven't received that sequence number yet but you have gotten packets around it. Have the client send a message to the host requesting to have those packets resent. There are a few ARQs that you can implement to do this. I did a selective repeat algorithm once that was pretty easy to implement.

Hope this helps.

Quote:

The UDP header is not visible to you at the socket layer

Sorry for beeing unclear, it was my header that I attach to every packet (32 bits of packet size and crc).

Quote:

UDP will not split a packet

Are you sure? That must indicate that my sockets are setup in the wrong way?
When I call send with 256 bytes for instance, it might return 64, making me calling send again? Edit: This wasn't actually the case :) It seems like the sendto didn't need to be called more than once for each packet.

Quote:

If you send "123 Hello, world", you will see zero or more packets containing exactly "123 Hello, world!". You won't see "123 " or ", world!".

That's what I was remebering from my old experiences.. this is good news!
I have to look very carefully on how I set up my sockets. I'll post the code later..

Quote:

For your question about getting things in order, how do you make sure you do that.

I have quite a nice system that seems to work fine (well, running the server on one machine and the client ond the other, but not vice versa). Basicly each peer have up to 128 "channels". Eavch channel can be setup in different ways (currently 2 flags = 4 ways). The flags are reliable and "skip old". Unreliable messages must fit into one udp packet.
Each channel for each peer has it's own 16-bit timestamp that is sent with the message. Messages that are reliable are resent after a given time, if they haven't been acknowledged. Messages going to the same peer are batched together into an udp-packet. All this seems to work fine. Unreliable messages are not acknowledged. Each message have a type associated with it: Begin, Intermediate, End and Single. Messages on unreliable channels are always Single. Messages on reliable channels can be: Single, Begin, End, Begin, Intermediate, Intermediate, End etc. Begin to End messages are always given ordered time stamps. All in all this seems to work just fine (even when simulating high packet drops, packet corruption (randomly modifiying packets), packet reordering and high-latency.


Edit:
Outgoing udp messages looks something like this:
[UdpHeader]
[MessageHeader]
[MessageBody (optional)]
...
[MessageHeader]
[MessageBody (optional)]

UdpHeader contains a 21-bit crc and size (actually you shouldn't need a size since that should be given from recvfrom?).
MessageHeader contains a 16-bit, message ID, a channel index, message size and message type.
[UdpHeader] is 32 bits in size
[MessageHeader] is typically 4 bytes in size (but can for long messages become 5 bytes).

Acknowledge messages is 3 bytes in size.

[Edited by - eq on May 4, 2005 3:02:54 AM]

I'm sending 24 byte packets (output from server via sendto).
The recvfrom returns the following sizes: 7, 24, 24, 16, 7, 3, 16, 24, 9, 32(?) and so on..

Could this have something do to with the fact that I'm using connection less sockets?

This is the source I use to create the server socket (error checking stripped away):


This is the source I use to create the client socket:


After creation of the sockets I just use recvfrom and sendto, correct?

One of the computers is a laptop using w-lan, can this cause trouble? (File/Printer-sharing works perfect between the 2 computers).
I find it odd that when I run the server on the laptop, the other computer get these odd packets. While if I run the server on the stationary, everything works fine? (Both computers sending and receiving data).

Quote:

You shouldnt have to do your own CRC because thats imbedded in the UDP functionality (incomming packets that dont match CRC are discarded automaticly).

Ok, will remove once the rest is working.

Quote:

If your protocol usage requires reliable delivery (cant handle missing data packets) you either need to implement a reliable protocol (ACKs and resends, etc..)

That's what I'm doing, all the packets that are sent/recv in the previous posts are NOT using this system however, it's perfectly valid for them to be dropped and delivered out of order.

Quote:

You are having a real significant problems if the packets you receive have shorter lengths than what were sent.

Exactly what my problem is!

Quote:

Check that its not a math error in the length you give the send call (or a error if packet has length imbedded in you own header...)

I'm quite sure the numbers are correct. I'm outputting the length parameter just before the sento call. If the returned number of bytes sent doesn't match that parameter I also output a warning (which I never see).
Then I output the size returned from recvfrom (if it's greater then zero).

Quote:

Original post by Anonymous Poster If you do get alot of lost packets (random, not too many sequentially lost) and your bandwidth isnt too tight, a trick used in some games is to piggyback the data from last packet on the next packet (effectively doubling the data sent). That way if one packet is lost, the next one will always have the missing data to patch the data stream. (Games frequently send many small packets of position/facing updates or status events and the extra bandwidth isnt that significant if you double the packet data from 32 to 64 bytes etc... it could also work with 1500 byte packets if needed 700 new data block + 700 last data block)

Thanks, but thats not the problem, I'm already doing this (sort of).

Does someone have a small client/server app that I can test on my configuration?
I'm starting to believe that something migh be wrong with the system, however I find it strange that http/ftp and filesharing work correctly.

I'd uploaded my application here and a version with more debug output here.

To start a server write Host.exe portnumber
To start a client connecting to a server write Host.exe ip:portnumber.

Samples:
Host.exe 6066 - Start a server listening on port 6066
Host.exe 127.0.0.1:6066 - Start a client, connecting to a local server on port 6066
Host.exe MyComputer:6066 - Start a client, connecting to MyComputer on port 6066

Hello eq,

sendto - when several calls are needed, sends pieces of the udp packet over internet?

I have found that depending on what type of system you use it could break apart the udp message into packets and send. Each packet is tag with a seq number so UDP layer can reassemble message. On the recv side you will get the whole message or none.
If one of the broken up packets of the message is lost you will not get the whole message.

If know on sun send a udp message bigger then 1445 it will break that message into packets of 1445 and send.
On recv side you just ask for message and you get it or not. It is handed at UDP to group udp messages back into order.

note: when send send with correct size and check for errors
on recv use max udp size 65535 for what you want to get and use size return for actual size of message received.

recvfrom - might or might not get these pieces?

true

recvfrom - can get the pieces in any order? (How the heck can one assemble the pieces into the original message if they only recieve 2 unordered bytes at a time?).

false, you get a complete message order or none at all.

sendto - can buffer some piece of data before it send it,

I belive false, each sendto sure send that message and not buffer like TCP will.

Lord Bart :)

Yippie! Things seems to work now. The scary thing is that I really don't know what the cause was (I think it was something with the dns look up and endianness that screwed things up, not sure).
I'd do some more testing before I'm confident that things work... and remove my udp header (size and crc).
I'm planning on keeping a 16-bit crc to verify my packets, i.e a system like:

On an udp packet send.
* (Optional) Compress packet.
* First 16 bits is the crc of the rest of the packet.
* The WHOLE packet is encrypted (including the crc) planning on using blowfish or something similar.
* Send encrypted data

On an udp packt receive:
* Recieve an udp packet.
* Uncrypt packet.
* Calculate crc, skipping the first 16 bits.
* Compare crc with the sent crc, if not equal, drop packet.
* (Optional) Decompress
* Process packet.

Is this a "good" way to encrypt a packet (preventing man in the middle attacks) and false clients connecting to the server.
The encryption key is never transmitted, the server admin can set the key. The user behind the client must get the key from the admin.
Does the 16-bit crc compromise a block encryption algorithm like blowfish?
I mean if a hacker knows whar crc algorithm I'm using and that the crc is stored in the first two bytes of the packet, is it easier to retrieve the key?