I'm currently trying to lay some code that will allow me to build an online multiplayer rpg.

I've pretty solid understandings of both newtwork protocols, (IP, TCP, UDP ecc) and of relational DB.

I'm in the process of building my own network protocol over UDP, following the gaffer on games tutorials, and as soon as I finish writing the protocol I will begin coding the API to interact with the DBMS.

I'm using C++, but not in a OOP manner, and I'm not using any framework or library: this way I have more control over what I do and I can learn much more.

I will probably use MySQL as DBMS.

Now coming to the game:

The game is 2D.

I've many different worlds procedurally generated, and the players will be able to move from one world to another in certain specific ways.

At the moment I don't know if there will be a limit to the players able to play in the same world, I will decide later, also considering the budget and the performance I get.

The world is fantasy, so it's populated by monsters, NPC, environment elements, ecc.

My idea about the client-server protocol is this:

The server will perpetually send clients data about their positions, what action they are doing currently, and what are the static and dynamic objects sorrounding them.

EDIT: another question. I would like to avoid sending the same object many times if it has not changed status, position ecc. For example, if there's a rock, I just want to send the rock position in one packet. I should be able to just send "world updates" to the client, right?

So the routine will be something like:

-for every player, the server checks what has changed in the world, and send those data to the corresponding client. Right?

(of course I will spatially partition them so the server has an easy life retriving the data ).

The clients will send packets when input occours and the player would like to send the request to make that move.

At the moment, my network protocol is "virtual connected", I also have sequence number, and I've implemented a simple "packet queue" that keep track of the packet that I've sended. So I've implemented some sort of "reliabilty".

Hopefully in the next days I will implement congestion avoidance and I will make some serious test with the protocol.

My questions are:

-regarding the network protocol:

is reasonable to let the application decide if the packet that it's sending has to "absolutely be delivered", or if the packet it's lost nothing occours.

For example: the packets the server sends to the client are not critical: if the server sends the client the packet number 99, and that packet get losted, well in the packet number 100 there will be the up-to-date positions, object status ecc. Is that right?

something like:

SendPacket( packet...)

ForceSendPacket( packet..)

Is a good idea to give the applications those API?

-If I want to be able to resend a packet, Is it best to keep the data at the transport level, (for example in my queue I also keep the data for all the packets), or it's best to communicate the application that "the packet number 99 has gotten lost, what do you want to do?"

(I think that this question depends on the answer given an the upper question)

I will implement a simple congestion avoidance method, something like incremental increase, multiplicative decrease: every time I get an ack, I icrease by a small amount, but every time I lost a packet, I reduce the speed by an half. Moreover, every time I correctly receive an ack, I will "take care" of the RTT, and adjust the sender speed. Make sense?

-regarding the "game":

I will surely implement some sort of client simulation, and then the client's action will be accepted of rejected by the server. But of course the server will always have the last word.

My question is about animation: is the server that sends client the current animation frame, right?

But this way the server will have to take into account the client simulation that has already happened.

Let's make a simple example:

-client's presses right button: the client starts simulating the movement, and set the current frame animation to "walk 1"

-the client sends server a message saying "hey, I'm moving right, Am I allowed to do that?"

-the server checks if the player is allowed to do that, and send a response saying like "ok, you can walk right, start the animation: walk 01"

-the client receive the message and set the animation to "walk 01".

But the client prediction was right, and some amount of time has passed: probably the animation should have been "walk 02".

What is the best solution to this problem? animations happens client-side? or the server takes into account that some amount of time has passed, and so it sends back directly "walk 02"?

EDIT: re-reading the post, I thought about a solution: The server ALWAYS keep track of what animation is doing the player, and what animation frame is currently on, so that all the player will have the same identical result on the screen.

To resolve the "startup" problem, when a client action is accepted, the server will assumes that the client has already started playing the animation, and so it will take that in consideration. But know I need a way to know how much time has passed from the start of the simulation? Could I simply say that "well, more or less the client is ahead half of the RTT with the animation regarding the server, so add half of the RTT to the "animation time". It's good enaugh? Better ways? I don't like the client sending the real time in every packet he sends...

I think the last solution is better, but I'm searching for confirm.

-considering that the game is 2D, and that I will use a tilemap for the terrain, plus every environment element (rocks, trees, rivers...) will have coordinates and status will I be able to have persistant world in your opinion? I mean, if I have a blueberry bush, and a player harvest it, I will be able to mantain that bush status over a server crush?

(I'm planning on having someting like 2000 * 2000 tilemaps, so I would say something like 2,5 millions of entities per world)

-regarding the DBMS:

Of course I will mantain the world status in the server ram, and I will update the DB asynchronously.

My critical tables are 2: the object table, of course, and the ability table, where I store which player has which ability, and at what level.

assuming that I will not make joins with these tables EVER, can I safely have a table with billions of entry? Better solutions?

Last question, very easy.

Let's say I cap the player maximum for "world" at 64 or 128.

But as I said, the player could change world several times/hours.

Is that a simple MOG or an MMO?

As always, sorry for my english, and thank you in advice.

Leonardo

First, any kind of "force" or "wait for response" function will end up blocking the UI of your game and be a terrible user experience.
Just Say No (tm) to blocking network calls!

Second, for an RPG, I bet that TCP will work well enough. That way, you don't have to worry about whether a client got the update that there's a rock in a particular place or not.
Just send out a packet whenever the client desires to do something (move to a place, move in a direction, cast a spell, whatever) and on the server, collate these and send to all clients X times a second.
For an RPG, I bet 10 times a second would be plenty.
Just make sure to pack the entire set of updates into a single frame/packet/send() call, and turn on SO_NODELAY.

If the client has the same map as the server, the client will 99% of the time predict correctly what the server does.
It can just go ahead and render whatever it it believes should happen.
When it receives updates from the server, it can compare that response with what it, itself, thought the state was at that time (the time the updated pertained to.)
It can then re-play whatever inputs the player provided between that time and now, to show the next player position.

For "important" actions, like the result of spells, the result of fighting, purchasing/transactions, etc, you should play a local "wind-up" animation when the player first initiates it, but you should wait to show the result until the server sends "fireball blasted here" or whatever back.
That way, you will never show the player "you succeeded" only to take it back again 300 milliseconds later.

Billions of rows in a database is generally a bad idea, not because you couldn't store it on a hard disk (a 4 terabyte hard disk can store a lot!) but because the height of the index becomes very tall. And one component of database performance is number-of-indices times height-of-indices.
For most game character data, I would just store the character stats/abilities as one big JSON blob that's stored as inline text (a long varchar or maybe text field.)
This reduces the index height (number of rows) to the number of characters, which is better.
If you really plan on having millions of players, though, at some point, you're going to want to horizontally shard your data -- characters with ID 0 - 999,999 live on instance A; ID 1,000,000 - 1,999,999 live on instance B; ...
Then keep a table of ID range to instance mapping, so you talk to the right database instance for the given customer.
You will of course need a central table that goes from "customer name" to "customer id" so that you know which database to look at; this may have to live in a central database, but should be a much smaller table with a single index.
If you have really quite a lot of customers, even the mapping from "customer name" to "character id" will be too big to keep on a central server; at that point, you may shard that initial table based on "first character of customer name" or "hash of customer name modulo 20" or whatever. But you'll likely never get to that point, as pretty much Google and Facebook have that problem :-)

Thank you for the quick and complete answer.

Of course the "forced messages" would have been those that don't need immediate feedback: login request, request to sell an item ecc...
Anyway probably it's best to go with tcp first, and then in case switch to tcp.

You're saying the server send updates 10 times per second... these updates contain also animation data right? If it so, it means that if my game runs at 30 fps I need to show every animation frame at least 3 frame... is that right? Isn't it a bit risky?

I will surely make use of the "fake" animation that you have mentioned, thanks for the advice.

Just to clarify, let me make another example.
-Client want to move right: the time is 22.00
The client start simulating the movement, updating the position, and show animation frame "walk right 1"
-the client send a message to the server containing the action (move right) and the time of the action (22.00)
-the server receive the packet, and checks if the client is allowed to move right.
If it is, update the player position, keeping in mind that the server time can differ from the client's one, due to the packet delay, so for example the packet arrive at server at time 22.03 the server has to account for those 3 minutes when sending back response to the client: if an animation frame lasts 2 minutes, the server will response "walk frame 2".
If the player wasnt able to do the move, the server simply doesnt take into account the player action, and it will send back the world status as nothing has happened.

Now the problem is: the client told me that it would like to move right at time 22.00, but when I receive the message my time is 22.03.
What do I do?


Going on to the database question,
Surely storing the abilities as simple text in the player table is an option, something simple like "fireball10 icecone5 heal3... but what for all the world object? Here I absolutely need a master table, right?

Something like:

ObjectID name position status

001 rock (20, 30) broken

002 magic sword (50, 10) full

003 sword (80, 20) broken

Of course, I Could drastically reduce the entry making the world not persistant, not allowing dropping objects ecc. But I would like to have persistant world also after a server crush or restart.
Any suggestion on this problem?
(Storing the player inventory as text is not an option, every equipment piece could have several magic abilities)

Maybe I could subdivide the object in various DB Instances?
For example objects of world from 0 to 100 are stored in an instance, objects of world from 101 to 200 are stored in another DB instance?

these updates contain also animation data right?

I don't think so -- you can easily calculate which animation (and frame) to play by knowing when an action started, and how long ago that was (in times.)

Your game could run at 144 Hz -- if it knows that the "fireball" animation started 1.3 seconds ago, it knows to display whatever frame is 1.3 seconds into the fireball animation.

Surely storing the abilities as simple text in the player table is an option

If by "player table" you mean the central table the just looks up from player name to player ID, then I'd avise against it, because you won't be able to shard that data.
Of course, if you never plan on growing particularly big, then any approach will work :-)

Regarding objects, you can still shard by ID -- objects ID 0 .. 999,999 live in shard A, etc. You then need a way to "reach" those objects. Easiest is to have an "inventory" table on the player ID shard.
So, the "inventory" table on shard A would contain the IDs of objects owned by players between 0 and 999,999.
Also, you'd need some "root IDs" for world objects, as well, if you give those object unique IDs.
(An alternative is to treat static objects that cannot really be manipulated as fixed objects that come from the level file, and have no real object ID.)
Depends on how much you want to be a "typical RPG" versus a "virtual world."

these updates contain also animation data right?

I don't think so -- you can easily calculate which animation (and frame) to play by knowing when an action started, and how long ago that was (in times.)

Your game could run at 144 Hz -- if it knows that the "fireball" animation started 1.3 seconds ago, it knows to display whatever frame is 1.3 seconds into the fireball animation.

Ok, so the server could just know when the player started walking, fishing, attacking ecc, update the world accordingly ( if a player fish for more then 3 seconds then spawn a fish). But I now need to send all the other players the "action timer", right?

So to any client I will tell:

player1 is attacking, the action started 3 seconds ago

player2 is fishing, the action started 4 seconds ago

player3 is walking right, the action started 10 seconds ago

...

this way the client will have the possibility to calculate all the other players animation status.

That seems very reasonable to me: this way, even if a client doesn't receive packets for a couple of seconds, it could animate other player anyway, because it knows what was the starting time of the animation.

But how do I resolve the problem of sync between client and server?

Because if a client send a packet at 22.00, that packet will take several ms to arrive at the server, and maybe at the time he arrives, the action is no more valid, even if at 22.00 it was.

For example, at 22.01 a rock magically spawn at the location player want to move.

even if the packet was sended at 22.00, the packet arrives at 22.01, and the action is not valid, even if at 22.00 it would have been.

Should I keep a buffer of world states in the server memory? one for 21.58, one for 21.59, one for 22.00, one for 22.01?

Other way to accomplish that? Could I just forget about "packet time"? (and of course consider only the packets within a certain "time range")

I mean, if the time server is 22.01, I accept packets that are 22.00, 22.01 and 22.02, but not packets that are 21.54, 22.08.

This something I'm curious about as well. Especially with movement, how do you properly synchronize the position between client and server? What I mean is, by the time "I'm moving" packet arrives to server client will have already moved a certain distance. Do you just design your game in a way that when the server corrects the position the client somehow interpolates the difference?

how do I resolve the problem of sync between client and server?

You have two main options:

1) Temporarily display the player in an incorrect/extrapolated position, and keep updating the position to be "more correct" based on what you receive from the server. This gives immediate command response on the client, but will display the client slightly out-of-sync with the world.

2) Only send commands to the server, and update the player to "walking" on the screen only when the server sends the new state back. This will always display a world that's in sync, but it will cause command latency between "start moving button pressed" and "actually starts moving on screen."

Exactly how you implement each of these options varies based on game mechanics and other specifics.
In most cases, the client won't ever be out of sync with the server, because there is no discrepancy.
Only when something is different (a door has opened on the server, not on the client yet; another player is in the way on the server, not on the client yet; etc) is the correction actually visible.
You can choose to lerp that correction, or just snap the player.
The simplest way to correct the player is to store the commands the player has received, and "wind them forward" each time you receive a new state update from the server.

player1 is attacking, the action started 3 seconds ago

Why would it take 3 seconds before you send out that information?

If you send information as soon as you get it - rounded to some sub-second granularity, at least - then you can just say "player 1 is attacking" and the chronological discrepancy will be so small nobody will care. (For RPGs, at least. Shooters are more demanding and require explicit trade-offs.)

Why would it take 3 seconds before you send out that information?

I think the point is that the server would keep sending updates to the state of objects.
Even if you heard 0.25 seconds ago that player1 is attacking, the action started 2.75 seconds ago, state updates allow you to stay in sync.
Also, if you support late joining, or interest management, then it's totally possible to hear about a character for the first time when it's in the middle of some long action.

so small nobody will care. (For RPGs, at least

Perhaps. Some action RPGs play almost like MOBAs, and there, the timing does matter.
So, exactly what your networking looks like, depends a lot of what your actual gameplay looks like.