Hey all, this is for a MUD, and I know that this doesn't really matter one way or the other. However, what do you think would be more efficient, using async sockets, or running several threads, with each thread select()ing on 64 sockets (or whatever the maximum select() can handle is)? Async sockets are supposed to be more efficient, but I can't help but think that if I have a few hundred clients connected, then I'll be getting a crapload of messages in my callback function. At least with threads and blocking sockets the congestion is spread out over several threads. I'm not at all concerned with portability, this'll be running on a Win2k server, and I'd like to support up to 500 simultanious connections. Cheers, Steve

If large numbers of connected clients is your goal, then the best solution by far is I/O Completion Ports. They can be a bit hard to get your head wrapped around, but once you get the basic framework up, they're actually very simple to code against. Certainly less code than having multiple threads listening with a select, and the state manegement is much simpler (in my opinion) than asynchronous sockets.

Hmm... I've played about with IO completion ports before (I wrote the previous version of my MUD using them, reasonable well), and there's something I don't like about them. Probably because I'm still trying to get to grips with them ;)
I seem to recall that async sockets could deal with up to 1000 connections, although that'd obviously be pushing things. Ideally I'd like to stay away from IOCP to be honest. I was thinking of 500 connections as an absolute maximum (after that I start refusing new connections with an error message). I'll be very surprised if the average load rises above 64 connections (1 thread).

However, I'd still like it to be as efficient as possible, but staying away from IOCPs.
If it matters (since I'll be running several threads), the server machine only has 1 CPU.

Some benchmarks of async sockets:
http://bulk.fefe.de/scalable-networking.pdf

Thanks, that's a very entertaining read :) Unfortunately, it only covers non-blocking IO with select(), not Async IO (The WSAAsync*() set of functions).

well, I'd probably say the management of the threads for the select solution is probably going to be more trouble than it's worth. For example, say you have 64 clients, then one more connects you start up another thread to handle it. Then one of the original 64 leaves. You've now got 64 connections again, but two threads. Do you shuffle things around so it's all on one thread again? If you have lots of joining/leaving there could be a lot of thrashing. You could do so that there's a bit of buffer space, and start a new thread when the "capacity" of your threads reach 80% or so, thus you can add/remove connections without creating lots of new threads. But like I said, it's probably more trouble than it's worth.

I/O completion ports work well because they minimize the number of context switches. And the OS is able to manage the number of threads in the pool.

Still, I think 500 connections will probably not overload a select solution too much. Plus, I think there may be a way to increase the 64 connections limit to something more reasonable (though non-portable, but if that doesn't matter...) I can't remember how exactly, but I think it was just a constant in a header somewhere..

Quote:

Original post by Evil Steve Thanks, that's a very entertaining read :) Unfortunately, it only covers non-blocking IO with select(), not Async IO (The WSAAsync*() set of functions).

Well, I'd probably stay away from WSAAsync*. They're mostly good for GUI apps that want to do socket I/O on the same thread as the UI. It's really only meant for one or two connections (i.e. a client app).

Quote:

Original post by Dean Harding Still, I think 500 connections will probably not overload a select solution too much. Plus, I think there may be a way to increase the 64 connections limit to something more reasonable (though non-portable, but if that doesn't matter...) I can't remember how exactly, but I think it was just a constant in a header somewhere..

Yeah, you can #define something before including the winsock header to increase the limit (up to a certain point). But that's also ugly ;)
If I was doing threads, I'd probably have some kind of loose thread pool, and try to keep as few threads as possible, but still keep the sockets balanced between the threads.
Hmm... You may be right about it not being worth it.

Quote:

Original post by Dean Harding Well, I'd probably stay away from WSAAsync*. They're mostly good for GUI apps that want to do socket I/O on the same thread as the UI. It's really only meant for one or two connections (i.e. a client app).

Really? I thought that WSAAsync was supposed to scale better than select()? Or it was better in some way. Perhaps I was thinking of something else.

I just remembered a chapter in Game Programming Gems 3 or 4 that had something about a very rough number of clients that each method could handle. I'll go and have a read at that and post what it says.

Thanks for all the replies,
Steve

Here's the paragraph (Game Programming Gems 4, Chapter 6.2):

Quote:

Clients: 500-1,000 Asynchronous and nonblocking I/O systems work well for 500-1,000 clients, but one-thread-per-client is unlikely to handle this many connections. Nor is it possible at this scale to have just one thread serving all clients, even with asynchronous I/O. Typically, servers for this model have 5-10 threads handling 100 clients each.

I think I might end up using multiple threads after all... :/

EDIT: Oh yeah, the #define is FD_SETSIZE, and it defaults to 64. I might bump that up to 128 if I go for the threaded approach I think. After all, if I do decide to go for linux (which I doubt), I'll have up to 1024 connections per thread (If I'm lucky).

Quote:

Nor is it possible at this scale to have just one thread serving all clients

I'd like to take exception to that particular statement. You can serve all the clients in a single thread just fine, given the proper software architecture.