Good questions. I'm not very far along, but answers are as follows:

1) did you use udp or tcp?
So far UDP only. My server code, so far, is pretty flexible though so I may look at adding TCP for certain messages (file transfers, etc.).

2) do you do any spoof protection?
I use IP/port to identify connections initially. I also check this against the clients unique ID. At some point I'll probably add a key of some sort to prevent simple spoofing.

3) do you use SSL for the login/password traffic?
No, I'll most likely use the MD5 method that RH is using. Currently it's just plain text over TCP.

4) Do you plan to store credit card numbers, and if so, how would you securely store it in your mysql database?
I'm planning on making my project free. If it ever goes commercial, I'll store CC# in a database (probably a different one from the primary database). On some of my web projects, I have encrypted them using simple encryption. This is worthless if your web server gets compromised, but if that happens, you're in trouble anyway.

5) how do all the various servers know which server is at what address? Is there a database for this or is it handled in startup scripts, or what?
Currently just one server. I'm not planning for a "massive" MORPG yet. I may once I get further along.

6) what platform are your servers running on? Linux/X86?
I'm using SDL_Net on Linux for the servers. The clients are DirectX 8 and also use SDLNet. The server code is all abstracted so if I want to use something other than NetSDL, I can easily replace it.

7) are you leveraging beyond2, nebula, nevrax nel, or twisted?
No. I've read a little about beyond2 in Massively Multiplayer Game Development, but it sounded complex and if anyone's going to complicate my code, it's going to be me!

8) what languages are you leveraging for development? All c++ or mixing in python, etc?
C++ exclusively. I may look at embedding python for some basic scripting. That sounds interesting.

9) what algorithm are you using for collision detection?
My collision is all very basic AABB collision at this point. Honestly, this may be sufficient for what I want to do. I'm going to focus on gameplay vs. realistic graphics/physics.

10) is your client directx or opengl or what?
DirectX 8. I'll most likely use 9 when I finish the prototype.

11) (Related to collision detection) How are you determining viewability for a client? As in, how do you decide what things the client can see from the server perspective?
Currently I'm not. My "map" is 2048 by 2048 so I'm just broadcasting everything to everything. This is obviously not going to hold up well under load, but it will be easy to add some simple range-based occlusion.

12) Are you using the observer/observable model for figuring out client updates?
Sort of. I think the pattern I am actually using is called "Multicast". Each message registers itself with a handler that extends an abstract IHandler. As the messages enter the queue, they are then passed off to the appropriate derived handler.

13) does the client send update positions once every 500 ms too? If so, how does this figure into determining if someone is hacking the client to walk through walls?
I'm working on this now so I can't really say. I'm hoping I won't have to send updates that frequently, but we'll see.

14) Is your client engine BSP based or outdoor roamish/portals based?
Strictly outdoor at this point. I'll most likely use portals when I start adding indoor environments.

My journal is back online if anyone is curious. It's goes back to the very beginning with screenshots all along the way.

http://www.bpopp.net/articles/view.php?id=403

[edited by - bpopp on November 16, 2003 3:57:54 PM]

quote:

---

Original post by red_mage
Your all UDP implementation inspires me to do the same. You're right, as long as you can apply a message sequencing/resend on top of udp, why bother with tcp?

---

Well, let me qualify that by saying you need to code it in such a way that you can apply it selectively to particular packet types. Also you should be able to send ordered OR reliable OR both depending on the need. If you don't do those things, you might just as well use TCP

quote:

---

">2) do you do any spoof protection?
Yes. My users are required to prove they are the person they say they are. This involves IP/Port checks as well as encoded ID values."
That is exactly what I am doing. Just curious, how big is your ID value? 16, 32, 64 bit?

---

32 bit. One somewhat unfortunate side effect of my security procedure is that if a dialup user (with a dynamic IP) loses thier connection and redials, they can't actually get back into the game, because they now have a different IP address than they logged into it with. They actually have to wait for the server to decide they are stale and auto-log off the account before they can get back in, which takes a few minutes. But, eh, what are you gonna do? I don't see any way around this without introducing more security holes than I have already, heh.

quote:

---

">3) do you use SSL for the login/password traffic?
No. I use MD5 encryption and the plain text version *never ever* leaves the client machine. Since MD5 is irreversable, I think this is secure (someone tell me if I'm wrong, heh)."
I don't think this is secure. What prevents someone intercepting a packet and reusing the hash to gain someone's login? True they won't have the plaintext password, but just resending the hashed to the server will work.

---

Yeah, that's a good point, and of course you're right. I'm actually in the process of adding a public/private key encryption system on top of protecting the plaintext password as well.

quote:

---

">11) (Related to collision detection) How are you determining viewability for a client? As in, how do you decide what things the client can see from the server perspective?
I'm using a landblock based system[...]
Just curious, how big are your landblocks, and how is the scale setup with respect to your player?

---

Of course, the scale is arbitrary. For various reasons, I've set a scale of 1 landblock = 1 sq. kilometer.

To be honest, I don't think my landblock system as it is right now will hold up for long. There could be literally hundreds of items in each landblock that would need to be transmitted every time a player crossed a landblock boundery. I have every item in my game indexed by landblock, so I can get at them very quickly, but even so, there is some overhead in putting together the packets and sending them.

On the other hand, what I *don't* want to do is check items against player for distance every time they take a step. That would be equally disasterous.

The problem is speed. Like someone said (I think it was KalvinB), if you spend an average of 1 ms processing each packet that comes into the server and you have 1000 players, you've just added 1 second of software induced lag to your game on top of the internet lag.

Probably what I'll do is some sort of hybrid between the two systems. I'll maybe to a check to see what landblocks are visible to the player, then check only those items in those landblocks (or subdivisions thereof) against the player to see if they are in range for sending their data. I may also split my landblocks into 4x4 grids for the purpose of item data checking/sending. I don't know, I'll have to see how it works out

quote:

---

hmm client driven updates. Thats interesting; it seems like a good idea.

---

Only way to do it, as far as I'm concerned. As far as packets go, the server should only respond, never initiate (as much as possible, sometimes you have to break that rule (chat packets for instance)).

bpopp, cool journal. It's always interesting to me to read about development progress!

-Ron



[edited by - RonHiler on November 17, 2003 2:23:26 PM]

Here's an update from my world...

Basically I'm thinking that maybe routing should be left to routing hardware. Real router/firewalls are going to give me the better price/performance than any software "smart router solution" IMO. They will *not* be as secure as my smart firewall/routers I described before. There are other problems as well.

The smart routers I had described required you to move ahead in the game login state before you could access the next server. In other words you could not physically get packets to the game landblock servers without first moving through the login and patching states. And once your game packets were being routed to a game landblock you still couldn't talk to the other game landblocks till you were handed off and rerouted to the next landblock server.

Another nice thing about them is they provided a single access control point for generating connection IDs, and only challenge/response for spoofing would need to be done once, instead of once for each socket that the client connects to.

Without the central connection ID generation of smart firewall/router method, the login servers will have to generate connection IDs for the client and broadcast the association to all backend servers. This is ugly.

The other real thing I have a problem with for just letting the client connect directly to the backend server is the client becomes responsible for addressing each landblock via its socket. I really do not like the possibilities that opens up.

I do like the smart firewall approach but I worry about all this smart firewalling being a network bottleneck. For each client the routers would have to store their external connection key, the internal connection ID and the accounting server/patch server/landblock server/whatever other server they are being routed to.

In the client=>server direction, the smart fw/r would need to translate external connection key to a client ID, then route the message to the listeners for this client id.

In the server=>client direction, the smart fw/r would have to translate a client ID to an (external connection key, client ip, client port.

For security, I would need to allow a max of say 10 connections from any one IP. This is to prevent flooders from taking my servers down by moving through the challenge/response to create the association over and over and eating up all my server memory. So I would need to manage this.

Plus I would need to manage a duplicate request for a connection from a particular IP/port. As in if the client didn't realize it succeded and tried rerequesting auth again. I wouldn't want multiple connection IDs mapping to a single client/port.

I would also need to time out the association for the client if they don't cleanly terminate thier connection.

What do you guys think?

Also I have given more thought to my class that will wrap around sockets. Its described in some detail here:
http://www.gamedev.net/community/forums/topic.asp?topic_id=191145

[edited by - red_mage on November 18, 2003 6:43:51 PM]

1) did you use udp or tcp?

- Both, I''m using pretty much the same design that Everquest uses, a central login server with multiple worldservers that are accessable from the login. Each "world" is comprised of multiple zone servers that all authenticate and register with the world server. Server to server communication is strictly TCP while the client to server communication uses both TCP and UDP depending on what server the client is talking to.

The client to server communication uses TCP if the client is talking to the login or world servers. Theres a breif overlap period where the client talks to the world server via TCP and the zone server via UDP, but only between the period where the client requests a connection with the zone server and the time the client gets back a response.


2) do you do any spoof protection?
- Not at this time. We''ll at least no intentional protection. MMORPG''s typically do not give out the clients IP address to other clients, so it would be hard for somebody to tamper with another''s connection outside of the local network. Inside the network, traffic could be potentially intercepted and modified but for the general user this shouldnt be an issue and a rare case.

3) do you use SSL for the login/password traffic?

- Nope. Right now the passwords are plaintext, but they will eventually be encrypted via a public key supplied by the server. RSA is fairly secure and should offer adequete protection.

4) Do you plan to store credit card numbers, and if so, how would you securely store it in your mysql database?

- Of course, the CC#''s will probabbly also be encrypted using some sort of asymmetric encryption.

5) how do all the various servers know which server is at what address? Is there a database for this or is it handled in startup scripts, or what?

- Basically the whole architecture is a tree. All the zone servers connect and register with their appropriate worldserver and the worldservers do the same with the login server. When the client wishes to connect to a certain server the client is sent the ipaddressort of where to connect.



6) what platform are your servers running on? Linux/X86?

- Currently developing on WinXP, once stable and theres no more design changes I''ll prolly port it to *nix.

7) are you leveraging beyond2, nebula, nevrax nel, or twisted?
- Nope, so far everything is homegrown aside from things like OpenGL and DirectX.

8) what languages are you leveraging for development? All c++ or mixing in python, etc?

- Its all C++. Scripting will prolly be handled by a custom scripting language.


9) what algorithm are you using for collision detection?
- None atm, low on the priority list atm while i finish up the network code and stuff. Its a toss up between dynamic plane shifting and elipsoid/sphere collision detection. Things are heavily leaning towards DPS since players of different races will be different heights, sizes, etc. It''ll prolly end up some combination of the two.

10) is your client directx or opengl or what?

- Both, OGL for graphics, DX for sound/input.

11) (Related to collision detection) How are you determining viewability for a client? As in, how do you decide what things the client can see from the server perspective?

- Right now its simply distance based. For the most part I''m keeping things simple my first go around.

12) Are you using the observer/observable model for figuring out client updates?

- I''m not familliar with how the observer design patter works. Positional updates are sent out every 500ms or so for close entities and right now its at 3 seconds for entities beyond the FOV of the player.

13) does the client send update positions once every 500 ms too?

- The client sends updates a little faster, usually when a key is pressed/released or when the player rotates his avatar. Idle rate is once every second.

If so, how does this figure into determining if someone is hacking the client to walk through walls?

- Haven''t adressed that issue yet, but its on my list.

14) Is your client engine BSP based or outdoor roamish/portals based?

- just a simple octree atm. Project hasnt progressed far enough along to worry about the rendering engine too much. In fact its last on my list of priorities. I''m trying to get as much of the other parts of the engine done before working on the graphics engine. I''ve got a simple renderer up but nothing elaborate.


-=[ Megahertz ]=-