Hello. I've been using this sample to learn how IOCP works and got most of it. Although that sample doesn't fully cover how would I shutdown the server. In some article it said something like "Post a completion status with NULL as completion key for each worker thread and in those threads exit if the completion key is NULL". Kernel will close socket handles upon termination of application but wouldn't locked buffers(WSABUF) be lost in this case? Is there any way to signal all I/O operations to complete immediately so that I can close socket handles myself and signal worker threads to exit if there're no open handles left?

Cancelling multi-threaded asynchronous workers is super tricky!

There is a cancellation function CancelIoEx which allows you to cancel outstanding requests. However, this has an inherent race condition in your application -- the request may complete right after you've called the function, but before it gets to the kernel, for example!

The best way to shut down these kinds of things, is to keep a "currently running" flag, and a reference count of outstanding requests. When you want to cancel, you do something like:

1) increment reference count
2) set "currently running" to false
3) decrement reference count

Each thing in your application that wants to perform I/O will:

1) increment reference count
2) if not currently running, decrement reference count and return
3) else issue I/O and decrement reference count in the completion function

Finally, the I/O completion function will:

1) decrement reference count

Now, the "decrement reference count" function will test whether the reference count goes to zero, and the handle is not running. That will allow it to claim "victory" in preventing more I/O from being issued.
HOWEVER: There is still a race condition, in that some I/O thread may be currently right before the "increment reference count" step, so after you claim victory, you have to then wait for all threads to complete, before you can finally deallocate the management structure.


If the only reason you want to shut down the service is because you want to terminate the process, then just call ExitProcess().
The kernel will clean up everything you've allocated and done (threads, I/O handles, allocated memory, open files, etc) -- that's the job of the kernel!
Simply set some flag that lets threads know that they should do nothing (so they don't keep going during shutdown and crash,) and then exit the process. The kernel is faster than you, and better than you, at cleaning up everything.

@hplus0603 Thanks for the answer. The sample I linked uses a similar approach to what you described with a counter variable for both outstanding and pending operations in each socket object(a small struct to keep client related information). What's the difference between calling ExitProcess and cleaning up memory/handles myself?

What's the difference between calling ExitProcess and cleaning up memory/handles myself?

It's easier to have bugs in your own code; the kernel is pretty well tested. But if you forget something, the kernel will do it for you anyway :-)

If you run debug code that asserts about ref counts on exit, though, then ExitProcess() won't let you test that code. The process just goes away. Which is fine by me.

It's easier to have bugs in your own code; the kernel is pretty well tested. But if you forget something, the kernel will do it for you anyway :-)

If you run debug code that asserts about ref counts on exit, though, then ExitProcess() won't let you test that code. The process just goes away. Which is fine by me.

Thanks I'll try to use "reference count" approach and see if it works. If not, ExitProcess is fine too as I'll be exiting application anyways.