Hello, I've been reading, and reading, and reading some more, and I'm hoping there are some knowledgeable people here who I can get into this discussion. I'd like to talk about large networking models, Server Client with lots of clients. Web servers, large chat programs, and MMOs (If you're insane). I just find this problem incredibly interesting. I've mostly been playing with C# and C++, though I'm not really looking for anything language specific... mostly theory. My first attempt was a pretty simple program which spun off a thread which housed a blocking listen socket on both the server and the client. Incoming packets were enqueued and then dequeued by the main thread, handled, then any responses were sent back through another socket on the main thread. The only thing my other thread was doing was retrieving the packet, and throwing it into a queue before it was back to listening. This to me seems like the most efficient method for anyone on a single processor box. Asside: My biggest problem in assessing the merits of these methods is that I am on a single processor box, and that my client program that I use to attack the program is ALSO on the same box. So they are all tryign to share the processor at once. Another way I have tried is as below... (All my code is simple C#) This method uses that Asynchronous BeginReceive(). There is also some timer code in there you'd have to take out if you try it. It also displays how many worker threads and IO threads you have allocated and available throughout the processing. This method uses the IO port pool, and even when I attacked it with 1000 packets from a client as fast as I could send them, it never dropped below 999, meaning it did them all with 1 thread out of the 1000 threads it allocated. Probably due to my 1 processor. This method seems like it would be a little faster than the previous method because it receives the data then re-queues the listen action before it actually does any processing with the data.
using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
namespace ConsoleTest
{
class ConsoleTest
{
private static int threads1 = 0, threads2 = 0;
private static Timer timer = new Timer();
private static bool start = false;
[STAThread]
static void Main(string[] args)
{
Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
IPEndPoint local = new IPEndPoint(Dns.GetHostByName("localhost").AddressList[0], 5050);
socket.Bind(local);
ThreadPool.GetMaxThreads(out threads1, out threads2);
Console.WriteLine("Max Worker Threads {0}, IO Threads {1}", threads1, threads2);
byte[] buffer = new byte[1024];
EndPoint client = new IPEndPoint(IPAddress.Any, 0);
socket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.None, new AsyncCallback(Listen), socket);
Console.WriteLine("Listening... ");
ThreadPool.GetAvailableThreads(out threads1, out threads2);
Console.WriteLine("Available Worker Threads {0}, IO Threads {1}", threads1, threads2);
while(true)
{
Thread.Sleep(10);
ThreadPool.GetAvailableThreads(out threads1, out threads2);
Console.WriteLine("Available Worker Threads {0}, IO Threads {1}", threads1, threads2);
}
}
public static void Listen(IAsyncResult res)
{
if(start == false)
{
timer.Start();
start = true;
}
Socket socket = (Socket)res.AsyncState;
// Start listening again right away.
byte[] buffer = new byte[1024];
EndPoint client = new IPEndPoint(IPAddress.Any, 0);
socket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.None, new AsyncCallback(Listen), socket);
// minic some sort of processing
for(uint i = 0; i < 4000000; i++)
Math.Sqrt(i);
Console.WriteLine("Done. Time: {0}", timer.Elapsed.ToString());
}
}
}
The other method I have tried is this one... This method doesn't use the IO pool thread, but instead it uses the worker pool thread. My computer reports it allocates 25 threads to the worker pool (much lower than the 1000 IO threads). However, it looks almost exactly the same in the end. You start the event function on a worker thread, in this case a blocking Socket.ListenFrom(...), then when it receives data you throw the Listen function back into the work queue right away just like the previous method. I clocked both of these last two methods to take just about the same time to handle the 1000 packet onslaught and that slowdown code I have in there. But as I mentioned above that's with the client on the same single processor. With this message I did manage to use up all 25 worker threads at one point, but usually it stayed around at 24 available. However, that makes me think that this method isn't as efficient since they didn't seem to have any time difference. If there were many more connections the Asynchronous IO method which used the IO thread pool might be better.
using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
namespace ConsoleTest
{
class ConsoleTest
{
private static int threads1 = 0, threads2 = 0;
private static Timer timer = new Timer();
private static bool start = false;
[STAThread]
static void Main(string[] args)
{
Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
IPEndPoint local = new IPEndPoint(Dns.GetHostByName("localhost").AddressList[0], 5050);
socket.Bind(local);
ThreadPool.GetMaxThreads(out threads1, out threads2);
Console.WriteLine("Max Worker Threads {0}, IO Threads {1}", threads1, threads2);
Console.WriteLine("Listening... ");
ThreadPool.QueueUserWorkItem(new WaitCallback(Listen), socket);
while(true)
{
Thread.Sleep(10);
ThreadPool.GetAvailableThreads(out threads1, out threads2);
Console.WriteLine("Available Worker Threads {0}, IO Threads {1}", threads1, threads2);
}
}
public static void Listen(Object obj)
{
Socket socket = (Socket)obj;
byte[] buffer = new byte[1024];
EndPoint client = new IPEndPoint(IPAddress.Any, 0);
int bytes = socket.ReceiveFrom(buffer, ref client);
if(start == false)
{
timer.Start();
start = true;
}
// Throw listen back into the queue before we use this worker thread to handle the data
// We can get a little more multi tasking now having it listening again.
ThreadPool.QueueUserWorkItem(new WaitCallback(Listen), socket);
// crazy delay
for(uint i = 0; i < 4000000; i++)
Math.Sqrt(i);
//string text = System.Text.Encoding.ASCII.GetString(buffer, 0, bytes);
Console.WriteLine("Done. Time: {0}", timer.Elapsed.ToString());
}
}
}
So those are the three methods I have personally tried, so far, myself. They are obviously very simplified and I'm hoping some people have things to add. The one problem I can see with the last two methods is scalability. You have a pool of threads, this is nice as they aren’t spun off as you need them and they don't sit idle if you have enough work for them, but still, you only have so many. In C# you can set the minimum number of worker threads, but I don't know if that will change after the pool is created (Something to test). I found a website
here Which is a C++ program which is a very scaleable version of IO completion ports (thread pools). From the quick look over I got from it, it uses the main completion port to assign tasks to the thread pool, but it scales as more people require it's use. there's also
this GameDev thread where a good handful of people go on about the horrors of multi threading. Some of it seems to be good. But some of it seems like a crock. If memory bandwidth is the problem then why would anyone bother with a multi processor computer if multi processing wouldn't speed anything up? It all comes down to efficient use of wait times... The questions are... does one thread used to block for connections work well enough to not bother doing anything else? Can windows completion ports more efficiently break up tasks than me defining a few threads and trying to efficiently manage their time myself? And what are the limits. Where should someone use one method over the other? Asynchronous IO ports allow you to send and receive data without blocking your thread, and while it waits you can process other parts of the program. This seems inevitably useful to me. But for most of us, I'm leaning towards a thread to listen, and MAYBE a thread for sending. // EDIT: Format
