I wanted to get some opinions on how to go about serializing floats.

Some people i've talked to have said to just assume that everyone is using the IEEE754 standard and just memcpy to bytes, but I didn't know how safe this was to just assume that the byte layout is the same especially between different operating systems such as Linux->Windows.

I had read this post https://stackoverflow.com/questions/3418319/serialize-double-and-float-with-c

and I saw a lot of different suggestions on how to go about serializing floats.

Do you all have any suggestions on the best route to go about serializing floats? Also, while we are on the topic, I am wondering the same about serializing doubles.

Note: I realize I could just convert them to strings already, but i'm specifically asking for methods that are not just converting everything to strings and sending it over.

From my limited experience, 32-bit floats are IEEE754 on all sensible contemporary platforms: x86, x64, PS4, Xbox One, all modern GPUs (!), ARM, ...

Mid-term outlook? Perfectly bit-wise compatible and fine to serialise. Should a problem arise, you can always re-code the serialised IEEE754 32-bit floats manually into a runtime format.

Can anyone mention a contemporary platform which breaks this?

Huh, funny, I just came across this post: http://www.gamasutra.com/blogs/AndrewErridge/20181004/327885/Building_a_Multiplayer_RTS_in_Unreal_Engine.php, with the following comment:

Quote

(2) Your claims about floating point non-determinism are a myth. The article you cite (and then those further citations) are myopic on the topic - none of those hypotheses were actually tested extensively on hardware. Floating point math is imprecise, but there is certainly nothing non-deterministic about the transistors living on your silicon chips. Yes, the IEEE standard does not require complete reproducibility. However, in reality - all commercial IEEE-754 FPUs calculate the same results when using the correct hardware modes and compliant FPU instructions with correct rounding. 

I know this because SAGE (the engine for Command & Conquer and BFME from 2000 - 2009) extensively used floating point math exchanged between Intel, AMD and even PPC CPUs in multiplayer mode and it used a peer-to-peer deterministic lock-step networking model that checked these floating point values for desyncs every frame.

So, it sounds like a non-problem, if that comment is accurate.

Determinism: Cross-platform floating point determinism is not guaranteed. AMD FPUs implement double precision differently from Intel FPUs, so even on x86, I've seen differences creep in. (This can be improved by controlling internal x87 precision bits and rounding modes.) SSE is in some ways more deterministic, but if you have libraries that take different paths for different CPU capabilities, you'll have a very tough time.

Regarding serialization, any platform you're likely to make a mainstream game for will use IEEE formulation for 32-bit and 64-bit floats. (16-bit may be a different kettle of wax, but that's mainly important for GPU shaders and machine learning models.) However, you may still find your code running on a big-endian system, especially if you want to run on more exotic hardware like embedded or IoT systems. Thus, for 99% of the time, you can marshal floats using memcpy(), and it will be fine. When you actually need to make sure, you should store the bits of the float into a uint32_t value, and use the htonl() function to put it into a known byte order and then go back using ntohl() and move it back to a float.

Another option is to know that your values only have a certain range and certain precision requirement, and marshal to/from fixed point. If your world coordinates are smaller than +/- 8192, and you only need three digits of precision to the right of the decimal point (say, one quantum is one-1024th of a meter) then you can get away with three bytes: int24 = (float * 1024) & 0xffffff Integer representations are more deterministic, may take less space, and usually compress better because the bytes of the value are more coherent to other similar values.

4 minutes ago, hplus0603 said:

Another option is to know that your values only have a certain range and certain precision requirement, and marshal to/from fixed point.

Quite so, real men use fixed point.

On 10/10/2018 at 12:10 PM, hplus0603 said:

Regarding serialization, any platform you're likely to make a mainstream game for will use IEEE formulation for 32-bit and 64-bit floats. (16-bit may be a different kettle of wax, but that's mainly important for GPU shaders and machine learning models.) However, you may still find your code running on a big-endian system, especially if you want to run on more exotic hardware like embedded or IoT systems. Thus, for 99% of the time, you can marshal floats using memcpy(), and it will be fine.

So much this.

If you are making one version of your game for sharing with the world, and you're only on the PC you don't need to do anything fancy at all.  Transmit the four bytes and be done with it. If you are sharing on multiple platforms that are using the x86 chipset (PC, PS4, XBOne) you can do the same. You will need to make certain your protocol works on all the devices, but can transmit the four bytes as-is for integers and floats and they are directly usable.

What you do with the data may have other issues and trying to build a deterministic game is a massive undertaking, but that's not the topic.  Serialization to the same chipset is not an issue.

When you change chips it is an issue.  Last generation of game consoles you would not have been able to share between PC and the consoles because they had different in-memory formats (in addition to the first-party companies rejecting the concept of cross-platform play).  Tools and data typically cross platform boundaries, so those needed to account for the different ordering between the PC where the data was built and the console where the data was consumed.