Hi guys,

I am trying to implement a network architecture in java for a small project etc., however I am unsure of what the design pattern/best-practice is for correctly identifying objects, which are coming through as bytes via DatagramPackets, so that I can then make the appropriate cast where necessary and reconstruct them.

The objects that are being communicated are various messages which will make up the client/server communication protocol.

I managed to dig up an old thread discussing this same issue and the agreed solution seems to be a dispatcher. Is this the best way of going about this?

I decided to give it a try as it seems like a decent solution. My message classes implement the Externalizable interface and I have roughly followed the design discussed in said old thread below:

http://www.gamedev.net/topic/634511-concept-of-a-basic-custom-packet-in-use-for-sending-data-to-server-and-client-back-and-forth/

In particular the last 2 posts (really 1 post but with the second-last post quoted):

I was worried that I have to write a few packets entirely in bytes. What a relief.

I don't think that's a bad thing. You can model the contents of packets as structs/classes, and then actually generate the reading/writing to/from byte arrays as functions, perhaps on a general interface that each of your messages implement.

reading/writing fields of a message isn't that hard to do manually, and gives you great control over how your protocol grows.

class WhisperMessage extends Message {
public int toPlayer;
public string text;
private static const int code = 10; // should not conflict with any other message class
private static const registration = new MessageRegistration<WhisperMessage>(code); // for finding class on receipt
public WhisperMessage(int p, string t) {
super(code);
toPlyaer = p;
text = t;
}
protected void writeData(ByteStream strm) {
strm.writeInt(toPlayer);
strm.writeString(text);
}
protected void readData(ByteStream strm) {
toPlayer = strm.readInt();
text = strm.readString();
}
};

Class Message would have public read/write functions that take care of also adding the type code and length fields to the output stream, as well as static functions to read message instances from a byte stream (you need to register the code with some dipatcher).

Also, please apologize if my Java is not quite syntactically correct; it's been several years since I last wrote any.

Better yet, make the Message class realize the Externalizable interface, which provide those public read and write methods and is seamless pluggable to Java's default serialization framework.

http://docs.oracle.com/javase/7/docs/api/java/io/Externalizable.html

I am however confused as to how I would implement the dispatcher (and to a degree the Message superclass - what does the constructor do with the code?) described in the second last post (I consider myself somewhat a beginner, especially with serialisation) and was hoping someone could elaborate?

tl;dr:

• What is the best (or at least an elegant) way of correctly identifying and reconstructing serialised objects?
• How would one link up the Message implementation, described in the post above, to a dispatcher?

Cheers,

Wulf

Generally all messages should have a header. If your game is TCP-based it can be as simple as a single integer that identifies the type data, if it is UDP based it will include the type of data plus a sequence number and potentially additional useful information.

You get to define what each of those looks like.

Personally I prefer the message format: { u32 size, u32 ID, u64 sequence number, payload }

Attempting to blindly cast the data directly to an object is perhaps one of the worst ways to handle deserialization. Apart from potentially transmitting unnecessary data, many languages and designs also have concerns about proper construction, proper alignment, data location, memory fragmentation, and so on. Also very important, you need to validate your data; you really need to detect things like an attacker sending an extremely long (or negative length) string.

So assuming the format above in a TCP message stream, you would wait until 16 bytes are available in the stream, and then extract them. The first for bytes are the size of the payload, the next four bytes are the way to interpret the payload, the next 8 bytes are the sequence number of the packet which is useful for debugging. Then you wait until the size of the payload is available in the stream. When at least that much content has arrived, you extract that many bytes and send the raw data as either a byte stream or byte buffer to a function that corresponds to the message number.

So let's say you got a message that starts out { 21, 10, 2471, ... } This tells you to wait until 21 bytes are ready in the stream. When they arrive they are packet type 10, which in your case might be 'player changed their name' message. So you dispatch it to a function. This could be through a big switch statement or through components registering their accepted message types with the system. Either way it ends up calling a function, perhaps OnPlayerChangedNameMessage( messageSize, payload ). Then inside that function you extract how long the new name is, validate the value (eg that player actually exists, positive string length, length matches the size of the payload, length within the size allowed by the game, etc.) and then extract the new player's name and validate it (only valid visible characters) and finally pass it along to the proper game component.

Does that answer the question well enough?

Hi frob,

My game is UDP-based, sorry I didn't explicitly mention that. I understand the theory behind network communication and agree with your idea of such a message format.

However, to my knowledge java abstracts that kind of lower level detail by providing network classes for both TCP and UDP-based communication. In the case of UDP, there is the DatagramPacket class which contains a byte array (of the payload) and length.

To reconstruct the byte array into anything meaningful (and following the Externalizable convention) I need to instantiate an object which will read the bytes before I can do anything else. See the following code:

	public void receive()
	{
		Message msg;
		byte[] buffer = new byte[1024];
		try {
			DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
			socket.receive(packet);
			msg = deserializeMessage(packet.getData());

			netMan.processMessage(msg);
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	private Message deserializeMessage(byte[] ba) throws Exception {
		ByteArrayInputStream bais = null;
		ObjectInputStream ois = null;
		Message temp = null;
		Message msg = null;
		try {
			bais = new ByteArrayInputStream(ba);
			ois = new ObjectInputStream(bais);
			msg = new Message();
			msg.readExternal(ois);
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			try {
				if (ois != null)
					ois.close();
			} catch (Exception e){
				e.printStackTrace();
			}
		}
		return msg;
	}

In this case I am instantiating a Message object, which is the highest level parent of all my different messages.

I have just thought of a solution where I continue (cannot start again as I have already read some of the bytes) reconstructing the appropriate message based on the code already read in the Message object. However, I am unsure if this is a good option as the parent Message class would be coupled with the message identification process...

I'm not a regular Java user, but have a quick read of the comments on ObjectInputStream here, and specifically the first example:

http://docs.oracle.com/javase/7/docs/api/java/io/ObjectInputStream.html

Note that they are not creating the objects or calling readExternal on them themselves. The are instead calling readObject on the ObjectInputStream which automagically returns the correct type. This assumes that you wrote the objects in on the other end using ObjectOutputStream, so that its metadata will be as expected. I expect you will end up with something simple like:

Message msg = ois.readObject();

Note that ObjectOutputStream/ObjectInputStream is essentially an implementation of what frob has described for you above, it's just they've done some of the work of reading and writing message types for you. Depending on your needs this might be OK, or you might want to get your hands dirtier and take full control of the data.

As for a dispatcher, that could mean lots of different things. This is a simplified version of how mine works in C#:

• All messages are derived from Message.
• The network service deserializes a message (as above) and verifies that it is valid.
• It then passes the message to the dispatcher, which in my case is called MessagingService.
• Various other parts of the system have registered themselves with the messaging service, specifying a message type (e.g. ChatMessage) and a callback (e.g. HandleChatMessage).
• MessagingService checks the type of the message, looks up the list of registered callbacks for that type, and calls them each in turn.

From there you can add whichever improvements you want. Some of mine include meta data on which client sent the messages, security information, and a bit of reflection to wire up my handlers automatically for me. My network is also registered with the MessageService so that the server can use it to post messages back to clients, without ever having to talk to the networking layer.

What this all gives you is a network and dispatching layer that is nicely decoupled from any details of your messages or what they are used for. It also lets your game or application forget about the networking side and just deals with neat strongly typed messages. My solution is more focused on convenience for the programmer that performance, but you get the idea.

Too much. You need a factory that can select the appropriate class based on the int type code. This class can instantiate the specific message class using a parameterless constructor. A switch statement is not a terrible way to go when selecting the class to create. Then the payload is given to the message class's deserialize method. That's all.

So one problem with using Serialization for a game is that it is difficult to force implementations on your users. Serial objects can be get very picky about what version of Java is running, and what version of class you are casting to/from. Imagine a situation where you've got to fix a bug, but that change breaks everyone's code unless they upgrade their Java or upgrade their game. I don't know about you, but I like to upgrade my stuff when I'm ready.

If you do the data yourself, you have complete control of updates, and the server can insert default data for new features, handle many versions of clients, many versions of code, and lots of other issues that will come up.

As always there are trade offs you have to consider for your own situation. I would consider a call to readObject to be much simpler and easier to maintain than a big switch statement. On the downside you a lot of control, and Glass_Kinfe notes some Java versioning issues I was not aware of.

For the record I work in C# and don't use the built in serialization, I have the equivalent of the big switch statement.

I want to note that readObject does not actually return the right type. Java has no support for dynamic return types, and readObject will just return an Object which you have to cast to the proper class through.. a huge ass switch statement (or equivalent construct e.g. factory pattern or an instance map).

For prototyping Java serialization is pretty awesome, but obviously for a robust project you'd want to provide your own serialization methods independent of the JVM runtime, as Glass Knife mentioned.

Correct, that would be the "dispatcher" originally asked about.

...a huge ass switch statement (or equivalent construct e.g. factory pattern or an instance map)...