Can I just get something clarified: Does async I/O mean using the WSAAsync* family of functions? And non-blocking I/O is polling with select()?

If that's correct, then is async I/O actually suited to handling up too 500-1000 clients, or is it geared more towards a few connections in a GUI app, as Dean said?

Non-blocking I/O means that you can recv() on a socket, and it'll immediately return "no data there" if there is no data. Using select() is optional with non-blocking I/O, although it's often a good idea.

Async I/O, on Windows, for sockets, means typically using the WSAAsync functions -- although any "overlapped" I/O operation would qualify as "async I/O".

I wouldn't worry TOO much about which method to use. Design a physical abstraction around network I/O, and implement whichever way feels most natural to you. If you have performanc eproblems later, you can go back and re-implent it in a better way without too much pain, assuming your abstraction actually serves its purpose.

Probably the one thread can't scale comment is referring to the fact that if your server only has one thread then it won't effectively utilize additional processors. The rule of thumb is one thread per cpu. How you get there can vary though. You could have 2-proc machine running two instances of a single-threaded server and get good scaling.

Quote:

if your server only has one thread then it won't effectively utilize additional processors

I would like to forward the argument that memory busses are more a limitation than CPU throughput. If you put two CPUs on a single memory bus, you'll just end up contending for memory, and may actually run slower than you would on a single CPU. If you are memory latency or throughput constrained, then multiple CPUs won't help you at all.

Ergo, it's often most cost-effective, AND highest performance, to design for single-CPU single-threaded nodes. If you want to scale, add more nodes.

Quote:

Original post by hplus0603 I wouldn't worry TOO much about which method to use. Design a physical abstraction around network I/O, and implement whichever way feels most natural to you. If you have performanc eproblems later, you can go back and re-implent it in a better way without too much pain, assuming your abstraction actually serves its purpose.

Yeah, that's true. I think I'll go for a multithreaded blocking socket method, and do a bit of profiling. I don't really expect the network IO to be a bottleneck until I have a lot of clients connecting anyway.

Quote:

Original post by Anon Mike Probably the one thread can't scale comment is referring to the fact that if your server only has one thread then it won't effectively utilize additional processors. The rule of thumb is one thread per cpu. How you get there can vary though. You could have 2-proc machine running two instances of a single-threaded server and get good scaling.

On a slightly-related note, the docs for IOCP (I can't be bothered to find the link just now) suggest creating 2 threads per CPU, I assume that's because the IO threads won't be doing anything for most of the time though, so the won't be much of a performace loss.

Thanks for the replies. I know that having a steady 500 connections really isn't going to happen, but even if I have 50 people on at once, I think that one thread per client may even then be a bit much.
Although, there's very little additional code and upkeep involved in extending that to x sockets per thread, so long as I don't get carried away (100+ sockets per thread would end up getting a bit silly).

About IOCP - I found it really weird to get to grips with, and I should probably go back and re-write my code, now I have a better understanding of how IOCPs work, since there's quite a few issues with my code. Perhaps I should have looked for a tutorial instead of reading the MSDN :P

Also, I really found that gem informative, thanks a lot for writing it :)

I advise just the opposite :). Thread-per-client is the worst way you can do things. Just wait until your server is acting weird and you have to wade through 500 threads to figure out which one is causing the problem. If you start a thought with "I'll just use a thread per-" and you end that thought with anything other than "cpu" then you've got a flawed design. Even if Linux manages the thread context switches better than NT then you've still got the fact that all those thread stacks are just chewing up memory that could be put to better use.

I find IOCP to be quite elegant. It neatly solves many of the scalability and thread-management problems for you. My understanding is that there is work to put something similiar to IOCP in the Linux kernel also but I don't really know.

My advise is to just stick with select for now. When you come to better grips with IOCP, and when scaling actually starts to be a real problem instead of a theoretical one, then switch over.

I've written a few muds for school projects and stuff and have used diffrent methods for network io. most muds that i have played rearly have 20 people logged on at once and most of the code bases that i've looked at stop accepting connections at 50. but most muds are written to run on a pentium 1 class processor and they still have latency with 40 players connected


the thing that is going to affect preformance the most is going to be how you design your mud to work with the connections and how you abstract your data whether it is io or player structures ect...

some of my test with my previous muds when profileing it at 30 fake connections only about 1% of my cycle was taken up by network stuff