Advertisement

IOCP shutdown design

Started by May 31, 2015 05:46 PM
17 comments, last by Taylor Softs 9 years, 7 months ago

I have a tcp acceptor class,

pseudo codes

class TcpAcceptor

{

SOCKET listener;

volatile unsigned int m_pendingAccepts;

void AcceptAsync(OverlappedAccept *ovl)

{

InterlockedIncrement(&m_pendingAccepts);

BOOL cbl = AcceptEx(listener, newSocket()....);

if fails call decrement

}

void Stop()

{

closesocket(m_listener);

m_listener = INVALID_SOCKET

InterlockedWait(&m_pendingAccepts, 0);

}

}

TcpAcceptor acp;

for (i = 0; i < 100; i++)

acp.AcceptAsync(new overlapped());

acp.Stop();

lets assume, program is executed closesocket function in Stop() (m_listener isnt set to invalid_socket yet), then another thread calls AcceptEx with that old handle (maybe anywhere in program would create new socket and having same handle with listener's handle, so erroneous implement)

I don't want to lock AcceptEx and Stop functions because I don't have any sync problems until I call Stop()

so how can I close listener socket and terminate accepts properly?

You either need some kind of interlock, or you trust the kernel to do the right thing.

The interlock way: First, tell each worker thread to stop working; wait for all the threads to complete; then close the socket.

The kernel way: Trying to do an operation on a closed/invalid handle will return an error. Detect this error and break out of the thread loop.

The second way is less elegant, because there is a small chance that the same handle value gets re-allocated by some other part of your process.

In typical server applications, you never actually need to close any handles, until you want to shut down the server, and the simplest way to shut down a server is by just calling exit()/ExitProcess() -- the kernel will close the files/sockets/other-handles for you.
enum Bool { True, False, FileNotFound };
Advertisement

You either need some kind of interlock, or you trust the kernel to do the right thing.

The interlock way: First, tell each worker thread to stop working; wait for all the threads to complete; then close the socket.

The kernel way: Trying to do an operation on a closed/invalid handle will return an error. Detect this error and break out of the thread loop.

The second way is less elegant, because there is a small chance that the same handle value gets re-allocated by some other part of your process.

In typical server applications, you never actually need to close any handles, until you want to shut down the server, and the simplest way to shut down a server is by just calling exit()/ExitProcess() -- the kernel will close the files/sockets/other-handles for you.

I just want to terminate listener socket and accepts because I'll recreate it. May be any risk-free implementation of kernel way?

Without any locks closesocket can be called on the socket before AcceptEx returns on another thread, no matter what you do.

AcceptEx can fail for other reasons than the listening socket being closed, so I don't really see the point, if you're trying to avoid AcceptEx returning an error?

You could write it so AcceptAsync never has to wait and only Stop waits if an AcceptEx is currently waiting to return.

I just want to terminate listener socket and accepts because I'll recreate it. May be any risk-free implementation of kernel way?


What do you expect the accepting threads to do in this case?
How do they get told about the new socket?
Why do you want to re-create the listening socket? If it binds to the same port as a previously listening socket, you may need to use SO_REUSEADDR -- but also, what would be benefit be?
enum Bool { True, False, FileNotFound };

This reply is really late, sorry.

I wanted to do this for perfect design. But even I ignore this issue, I'll meet similar problem when aborting peer socket.

Assume, we have 2 threads, 1 socket and 2 overlapped operations.

1.Thread closes socket and 2. thread calling wsasend, so maybe socket handle going to be reused (whilst handle can be allocated again) as you described.

1. Thread

int ret = WSARecv(sock, ...)

if (ret == socket_error)

{

closesocket(sock);

sock = INVALID_SOCKET;

}

2. Thread

WSASend(sock, ...)

I don't want to use lock since WSARecv and WSASend can be called simultaneously. It would degrade performance, I believe there are ways to synchronize it without locks.

Another question is,

If I set linger struct members on=1 and linger=0 on socket, DisconnectEx(TF_REUSE_SOCKET) would do hard close like closesocket does or still graceful?

Advertisement

1.Thread closes socket and 2. thread calling wsasend


This is a bug in your program.

If your program is structured such that this can happen, then your program has a bad connection management model. The only reason to call close() on a socket is if your program has determined that it has no more data to send or receive on the socket. Typically, this will be because the protocol has executed some "end of transmission" handshake.

In that case, the threads that do sending/receiving will detect this, and remove the socket from the pool of active sockets that get worked on, and then put it in a queue of sockets that should be closed.
enum Bool { True, False, FileNotFound };


The only reason to call close() on a socket is if your program has determined that it has no more data to send or receive on the socket.

If I have packet receive filter (e.g. 10 packets in one second) and peer exceeded it, I want to close socket hard and abort all operations from this user. So how can I do this?

Can you recommend me some IOCP servers(to be used in chat apps, mmo, file transfer etc) source codes to dig out? and some libraries to imlement IOCP in painless way?

If I have packet receive filter (e.g. 10 packets in one second) and peer exceeded it, I want to close socket hard and abort all operations from this user. So how can I do this?


If you receive multiple messages at the same time from the same remote end, how does your program synchronize that?
Synchronize this decision in the same way.

some libraries to imlement IOCP in painless way?


We are talking about asynchronous, multi-threaded programming here. There is nothing "painless" about it. That's why this is a hard area. It's totally doable (in the sense that computers are deterministic,) but it does require skill, determination, and measuring what you do.

That being said, I typically like the boost::asio library, because it has the same interface on multiple OS-es, and is reasonably efficient on each of them.
enum Bool { True, False, FileNotFound };


If you receive multiple messages at the same time from the same remote end, how does your program synchronize that?

No, I don't have. I meant If I receive more than 10 packets in one second from peer (with single outstanding WSARecv), I want to close socket hard immediately.

This topic is closed to new replies.

Advertisement