Hi guys, I''m working on my UDP WinSock IOCP server, and it''s going pretty well, but one issue kind of confuses me. It has to do with the input and output buffers. I was going to use a one client/socket approach for outgoing data (i.e. when a client logs into the server, they are assigned a socket, which is "connected" to their system via the WSAconnect() call). Each client gets their own 2K buffer for outgoing messages (from server to client). This would be enough to store at least one or two messages. Is that enough, or ought I to raise that a bit? If a completion notification hasn''t been received for the first outgoing message, and I need to send another message to the same client, I have to be sure the buffer remains intact for the first message, yes? With 2000 clients, 2K per client is already at 4MB of buffer space. I can''t raise it too much more, can I? For receiving messages, I was going to use a single socket, with a large buffer (32K?), the idea being to store incoming datagrams in one big pool until I can process them, then remove them from the buffer as I get to them. The problem I''m having is with that large buffer. Suppose I have two client giving data to the server. With the WSARecvFrom() call, I have to specify a buffer, so I pass the address on my large buffer (actually a start address, a pointer to somewhere inside the large buffer). But wouldn''t the two client''s incoming data overwrite each other? Of course, what I want is for them to write to the buffer at different locations so they don''t overlap. Now, by using only a single socket for incoming data, can I be sure that if several clients are sending data at the same time, I am only going to get one at a time? Thus I could update my "start address" pointer into the large buffer before the next datagram starts getting written to the buffer? A related question. Suppose I have 2000 clients connected. Is that one input socket going to be a bottleneck? Do I need to have more than one input socket? Whew, lot of questions. Comments are appreciated, both on the questions and the basic scheme. Ron

Most routers like UDP packets of no more than 1400 bytes (MTU). Above that they either cut them to pieces or forget about them altogether. Keep that in mind when you design your message protocol. As for buffering, I suggest you build each posted call a buffer of its own that you can coalesce later into bigger ones in the IOCP thread if you need it to. That per-call buffer should be from a larger contiguous memory pool rather than using malloc/free calls; the kernel can lock down your per-call buffers in memory until the posted call is complete. In the same vein, the OS''s MMU granularity is 4K so make your buffer on a 4K boundary and size modulo-4K if possible; otherwise you might get locks all over your application memory and the system will start swapping.

Having separate sockets for incoming and outgoing traffic will get you into trouble with NATs. It''s better to use the same socket, especially if it''s the client behind the NAT that initiates the communication and your server is directly on the net.

> Suppose I have 2000 clients connected.
> Is that one input socket going to be a bottleneck?

You seem to imply that your server would get overwelmed with incoming and outgoing traffic, hence the radical design; maybe I''m wrong. Unless you have multiple network cards in your machine or you have an FDDI- or Giganet-type card, your main CPUs will always be a lot faster than the I/O processor on the network card. It all boils down to how much processing each IOCP thread has to perform wrt the main thread: just copy buffers around, do disk I/O such as fetch an HTML file or do an SQL query, or perform some complex simulation. I suggest you do the math yourself: 10 or 100Mbit network card or local router, 8 bits per byte, 32 - 56 bytes overhead per message (depending on support), X bytes per message average = how many messages per second.

-cb

quote:

---

Original post by cbenoi1
Most routers like UDP packets of no more than 1400 bytes (MTU). Above that they either cut them to pieces or forget about them altogether. Keep that in mind when you design your message protocol.

---

Right. My packet data doesn''t get bigger than 1K, so I think I''m okay there.

quote:

---

As for buffering, I suggest you build each posted call a buffer of its own that you can coalesce later into bigger ones in the IOCP thread if you need it to.

---

Okay done. I was coming around to that conclusion myself I''ve changed my scheme to use 2000 2K buffers for both input and output, with an operation flag.

quote:

---

That per-call buffer should be from a larger contiguous memory pool rather than using malloc/free calls; the kernel can lock down your per-call buffers in memory until the posted call is complete. In the same vein, the OS''s MMU granularity is 4K so make your buffer on a 4K boundary and size modulo-4K if possible; otherwise you might get locks all over your application memory and the system will start swapping.

---

Okay, will do.

quote:

---

Having separate sockets for incoming and outgoing traffic will get you into trouble with NATs. It''s better to use the same socket, especially if it''s the client behind the NAT that initiates the communication and your server is directly on the net.

---

Hmmm, not sure I understand how this "one socket" idea would work. If I post a send call on a socket on the server, what happens if 10 clients then post 10 SendTos to the server? The socket would be in the middle of an send IO, and wouldn''t be ready for the 10 receives. There wouldn''t be a socket with a pending WSARecvFrom() call available. Wouldn''t the clients error? Or worse, just have their packets get lost?

That''s why on the server I was going to have a single socket for sending out data (I can buffer sends until the socket is ready for each one), while giving the clients each a personal socket for receiving data on the server.

On the client side, I''m using one socket for everything (receive and send), but I''m not using IOCP there (I''m using Asynchronous sockets). Come to think of it, that might cause a similar conflict.

But perhaps I am missing a fundamental concept. It''s very possible, this is my first IOCP Would the incoming receives hang around somewhere until my single socket was ready to read them (after it was done with the sending).

quote:

---

> Suppose I have 2000 clients connected.
> Is that one input socket going to be a bottleneck?

You seem to imply that your server would get overwelmed with incoming and outgoing traffic, hence the radical design;

---

Hehe, well the radical design is more likely due to my inexperience in this area. I take your point, the one socket would work fine for incoming data.

> Hmmm, not sure I understand how this
> "one socket" idea would work.

Let's get back to the 2 socket solution for a second. The client connects to a socket, say on port X on your server; the client's sending port number is W (either you bind()'d it or you've let the OS decide for you). You create a secondary socket for this client on port Y and send back a reply to the client on port W. The client's router saw the following connection go out:

W ==> X

and records it as valid but never saw any traffic pertaining to

W ==> Y nor W <== Y

and can thus conclude this is a hacker's attack on the port and shut down the W<==>Y communication link, even though it comes from the same IP and application! That's what was meant by "...will get troubles with NATs"...

> But wouldn't the two client's incoming
> data overwrite each other?

recvfrom() and WSARecvFrom() both give you the IP/Port of the incoming packet's sender. Sockets have a small buffer located in the device driver; if you turn it off, then yes, you have to make sure you extract the incoming data fast enough. This is not a problem with overlapped/IOCP because you provide this lockable buffer yourself. Here is something you must read to get a better understanding:

http://msdn.microsoft.com/msdnmag/issues/1000/Winsock/default.aspx

especially the section entitled "Who Manages the Buffers?".


> The socket would be in the middle of an send IO,
> and wouldn't be ready for the 10 receives.

Consider a socket like a pair of one-directional sockets, each being able to operate independently. The only inter-dependence is sharing of the hardware protocol (*). Consequently, you can devise different strategies for input and output, i.e. IOCP for reading and polled non-blocking for output (**).

-cb


(*) In general, you can't send a message when there's an incoming one on the wires and vice-versa, regardless if the message is a UDP, TCP, ICMP or other type of packet; that changes when the packet reaches the 'net routers and you can get radically different route for incoming and outgoing packets. Your network card and your local router are the real bottlenecks.

(**) You need Winsock 2.0 or better for this to work. You can't use select() on read-ready while writing (or the other way around) under 1.X. Somehow MS got the 1.X socket HANDLEs screwed up under a variety of multi-threading scenarios.

[edited by - cbenoi1 on May 24, 2003 8:15:57 PM]

Thanks for the link. It was a good read.

quote:

---

The client''s router saw the following connection go out:

W ==> X

and records it as valid but never saw any traffic pertaining to

W ==> Y nor W <== Y

and can thus conclude this is a hacker''s attack on the port and shut down the W<==>Y communication link, even though it comes from the same IP and application! That''s what was meant by "...will get troubles with NATs"...

---

Errr, but what prevents me from using port W on the server and port X on the client exclusively? i.e.
W <==> X
This is in fact what I was planning on doing, even though I''m using different sockets on the server for sending and receiving from that client. This should not cause problems with the NATs, right?

So your next question is going to be: "If you are always sending and receiving on the same port, why bother creating all those sockets, why not use one socket for sending and receiving to *all* the clients".

Again, I come back to the same issue. Perhaps this is a faulty assumption, but I''m just not comfortable using a single socket to handle 2000 clients (I use 2000 as my worst case scenario). But the more I think about this, the more I think that perhaps it is not necessary to keep a socket pool, that I can run everything through a single socket.

Do you really think the one socket can handle both input and output for 2000 players at once without losing data or becoming a bottleneck? To be honest, I still worry a bit about what happens if a client posts a WSASendTo to the server, but there is no socket with a pending WSARecvFrom in the works (which could happen in this scenario when the completion post of a WSARecvFrom is being handled). Should I perhaps post multiple WSARecvFrom calls on the socket to be sure there will always be one pending to accept datagrams, or is this not necessary? Can I even do this (I haven''t tried it)?

I will think about this. I''m not *quite* to this point yet in the programming (I''ve finished initialization and posted an initial WSARecvFrom, I''m just now starting to work on the worker thread). So basically, I better make a decision on this pretty much right now

> This should not cause problems with the NATs, right?

Svr W <== Clt X was seen by the NAT and thus Svr W ==> Clt X will pass through. You can't bind() more than one socket on the same port; so can't have Svr W1 <==> Clt X and Svr W2 <==> Svr X and pretend it's the same 'Svr W' source port.

> Do you really think the one socket can handle
> both input and output for 2000 players

What is a socket? Like a file handle, it's an OS-made representation of a communication link, and contains buffer areas and a state. Having 2000 sockets around won't suddently make your network card go 2000 times faster. Having 2000 handles opened on a disk file won't make your file load quicker nor make your hard disk spin faster either. On the contrary, you are making the OS sweat bullets in managing all those 2000 sockets for you; the network card's payload is still the same.

Having multiple sockets is not a bad thing in itself. Sockets allow you to assign different meanings and priorities to the communication links. You could have a low-priority UDP socket thread solely for login/logout and other account transactions, and another high-priority one for in-game data exchange. TCP also lends itself to a one socket-per-client topology because of its state-based nature.

> The problem I'm having is with that large {32Kb} buffer.

Here's a quick test: create 16 IOCP threads with each its own 2K region of the buffer (since each message is at most 1400 bytes there are no collision, right?). After a thread is activated and the incoming data processed, repost the RecvFrom with the very same location in the big buffer. Count how many times each thread is actually triggered. Depending on the complexity of the data processing step, you may observe that many threads are never triggered at all.

-cb





[edited by - cbenoi1 on May 27, 2003 10:12:32 AM]

Okay, cb, you convinced me (again)

The scheme I am going with is a single socket, using a pool of 2000 2K buffers for input and output. On the advice of the gang at alt.winsock.programming, I am posting an initial 10 WSARecvFrom''s (apparently I *shouldn''t* lose any data if a datagram comes in when there is no WSARecvFrom pending, but it is possible if the socket buffer fills up, and in any case it will slow down the data buffering process if that happens). Since I already had initialization code in place, finishing this up was pretty easy, and everything seems to be working fine.

I''ve added the ability to monitor my buffers (I have four states any buffer can be in: Available, Pending Read, Pending Write, and Finished Read (which just means I''ve gotten a completion notification on that buffer that was in pending read mode, and I''ve flagged it as "input to be dealt with")). After initialization, I have 1990 buffers available, 10 in pending read mode, and 0 in the other two, which is correct.

The real test will be when I start sending datagrams from the client, to see how the buffer distribution moves around. As long as the Available buffer count never gets too low, and the Finished read doesn''t get too high, I think I have my basic networking system done. Woohoo!

Thanks a bunch for the advice you''ve given. IOCP is not an easy thing to learn, and I hope I wasn''t too obnoxious with all my questions

Ron