Hey everybody. I have been having some issues learning IOCP and I was hoping someone here might understand it well enough to help me out.

I'm currently trying to learn the basics of utilizing IOCP from this code sample. Note: This sample is not using AcceptEx but will look into that later.

https://github.com/Microsoft/Windows-classic-samples/tree/master/Samples/Win7Samples/netds/winsock/iocp/server

This is an echo iocp server from Microsoft.
There are a couple things I am confused about.

When a new connection is accepted, a WSARecv is called and the overlapped structure is passed in.

From what I gathered, for this particular code, in one of the worker thread's, the  Worker Thread Function will be able to then receive the data from this WSARecv call via the call to GetQueuedCompletionStatus.

From the Worker Thread Function, it looks like this server follows the following pattern per connection:

1. Check if there is a pending io operation for some connection

2. If pending io operation is a read operation, store the size/buffer in the overlapped context and queue up a WSASend call as a write operation using the overlapped structure.

3. If pending io operation is write operation, check if the # of bytes matched the total bytes. If all bytes were sent, update the overlapped structure and queue up WSARecv for next read operation. If not all bytes sent, queue up another WSASend to attempt to send the remaining bytes.

So it seems like the pattern is this server attempts to read data, then will echo that data back and once that data is echo'ed back, the server will read data again.

Here is where I am confused.

How would I handle the following situations:

1. In this situation, I want the server to continue performing as an echo server. On top of its current functionality, I want it to also dispatch a 4 byte message every X seconds.
2. In this situation, I want the server to echo all of the client's messages to the other clients as well. So if client #1 sends 'AABB', the server will send 'AABB' to client #1 and the server will send 'AABB' to client #2.

Let me explain why I am stuck on these situations.

For situation #1, The server is not guaranteed to send the full packet when echo'ing data back to the client. How would I guarantee that the server will echo back the full packet before dispatching the 4 byte message? I considered having a queue or something tied to the socket context, but I am also wondering about how would I send the 4 byte data if the client never sent anything to be echo'd? Will it be an issue if I have queued up a WSARecv on the socket, no data has been received, and I queue up a WSASend at the same time? Also, the call to WSASend to dispatch this 4 byte data will cause an extra call to WSARecv after the data is sent. How should this be avoided since there will already be an outstanding WSARecv from the server listening for packets to echo.

For situation #2, my confusion again comes across verifying that the packets will be sent in the correct order and not split up in case of partial sends and a similar confusion about if this new call to WSASend will cause an extra call to WSARecv after the data is sent.

Sorry the post is so long. I tried to include hyperlinks to the code in question to clarify what I am asking. Thank you so much for any input.

You have this same problem in a non-IOCP server, too! The main thing that changes with IOCP is how you structure your threads (one worker thread per CPU,) and how you structure your application to know when to do what (basically, build a state machine.)

There are many structures that can make sense. The main trick is going to be building something that is thread-safe enough for multiple completion threads to run simultaneously, yet doesn't serialize all logic on a single lock.

One way to structure a server like this would be:

1. There's a global hash table from "socket handle" to "client object." Inserts/removes/look-ups in this table need to be thread safe, but once you have the client object, the hash table doesn't need to stay locked.

2. The client object has an "incoming buffer" array of bytes, with a "head,"  "received," and "parsed" pointers (for where it's attempting to receive to, what has been received already, as well as how far it's managed to parse.) The buffer is treated as a cyclic buffer. The client object also has an OVERLAPPED structure to use for receiving into this buffer. As long as there is space in the in buffer, there should always be an outstanding receive call trying to receive into the empty space of this buffer, so each time a receive is completed, you should start another one (if there is enough empty space left,) then start parsing whatever you have.

3. The client object has another OVERLAPPED structure, plus an "outgoing buffer" array of bytes, with a "should write," "currently attempting to write" and a "written completed" pointer. As long as there is any data to write, you should have an outstanding write request, so when one completes, if there's anything in the buffer, start a new one. If something is added to the outgoing buffer, check whether a request is outstanding, and if not, start one. Note that you have to manage the "request outstanding" state yourself (with a flag or something,) because otherwise you will race with the OS.

4. The operation of "put stuff into the incoming buffer" must use a lock that's specific to the object, so each object probably has a lock (a CRITICAL_SECTION for example.) "Deal with data that was received" doesn't necessarily need a lock, if you "deal with data" before you "queue another read request." If you queue the read request before parsing the data, there's some risk that that second request will complete on another thread before you are done dealing with the data, so in that case, you also need a lock for the "deal with the data" read case. You'll typically also want to put a length value (say, 2 bytes, for a max length of 65535 bytes per packet) into the outgoing buffer before the actual bytes of the message.

5. "deal with the data" typically looks something like: Do I have less than 2 bytes? Just wait for more. Else, interpret those 2 bytes as a packet length, without consuming them. Do I have that many additional bytes? If not, just wait for more. Else, call some function to deal with a "complete, received packet, of size X" and then advance the "data read (buffer free)" pointer, and then go back to check whether you have another ready packet.

6. The handler for dealing with a packet may decide to forward the packet to other clients. To do so, iterate through the table of all clients, and call the "enqueue data" on each of those clients. The iteration over the table of all clients will need locking that table; calling write on each client object will lock each of those objects; there is risk for deadlock here if you're not careful!

7. You may run into the case where a client doesn't receive data fast enough, and there is no more buffer space left. You have the option of dropping the message and writing code that deals with not all messages getting to all clients, or detecting that the client can't keep up, and kick the client from the server. Those are your only two options. Specifically, you can NOT wait for the client to drain, because this will block a handling thread for an unknown amount of time -- potentially forever if the client is a suspended process that still responds to TCP ACKs but doesn't open the receive window. I recommend reasonably-sized buffers (256 kB per player?) and kicking players that don't receive their data fast enough.

8. Your "every so often" periodic message can be handled in a few different ways. One is to have a timer in the IOCP list of handles, and when it expires, iterate through all clients. Another is to have a separate thread which sleeps, and when it's time, wakes up, and iterates through all clients. A third is to have each client have a variable for when they last had one of those messages, and each time you receive something from the client (in the receive handling function,) check whether the time is right, and generate the outgoing packet and update the variable.

As you can see, there are many, many, bits and pieces that need to go together "just so" to make a high-performance, correct, asynchronous networking server. You need to understand locking, threading, cyclic buffers, contention, TCP protocol details, timing, and a number of other concepts. If you can learn and apply all of these, you can build a beautiful, event-driven server that makes very good use of available machine resources. If you're new to all of these concepts at the same time, you will probably find that trying to climb 8 different walls at the same time will be quite a challenge.

Good luck on your project, and make sure to keep us up to date on what you learn!

Thank you so much! What you've said makes a lot of sense.

I've written basic servers using nonblocking sockets over select before, but getting into IOCP is a big jump for me.

I'm going to apply your advice and I think I have a good idea of what direction to take now.