This is a little tricky to explain, please change the title if you feel it does not truly describe the problem.

Background: I'm working on a resource loading and saving mechanism for my game engine. I have a class called a data bundle which stores a list of sprite-sheets. Sprite-sheets store the actual pixel data values of a “png” image, its width and height (This cannot be changed as the rendering system requires this data in this format).

The data bundle can write itself to a json file which contains the data from each sprite-sheet in its list. This is all fine but then I figured what if I was able to some how encode? convert? pack? the data values in the pixel array into some data structure such that if it were written to a file, the data points would be less? When I read this file I should be able to convert that data in to the original pixel data.

Is this even possible and how necessary do you think this might be?

If this is not advised please say so but, if this is a possibility please let me know.

Note: I am able to produce a compressed gzip file which I can read and write to, perhaps this is good enough?

Ninja Boss Fight said:
If this is not advised please say so but, if this is a possibility please let me know. Note: I am able to produce a compressed gzip file which I can read and write to, perhaps this is good enough?

I'm only using a compressed file for all the assets in a “block”, this is good enough for my needs. Even though the game only has 10MB of texture data, I made different experiments and concluded that using specialized compressed formats inside a zipped file didn't make any difference. Using this approach has the benefit of being able to load the texture data directly into memory in DirectX without having to further decode/decompress it.

One technique worth mentioning that old games used is called “palettes”. You create an array of all the colors in the image, and instead of storing per-pixel color values for the image itself, you only store the (8-bit, potentially smaller) index into the palette. This of course only really works well when you don't have many different colors in the image, and/or large images (so that the size of the palette is offset by the size saved for each pixel), but this could work in combination with your sprite-sheet, don't know for sure.

Consider that PNG images are very well compressed: if you are already decoding them one by one and the only other data is the JSON index of spritesheets your data size should already be very close to optimal, don't invent anything complicated.

However, the PNG files themselves could be optimized for smaller size if you care:

• Consolidated large sprite sheets instead of many small ones: to have less files (less index entries, less headers, less chunks of the PNG files that are the same for all sprite sheets) and to help compression (amortizing the “warm up” of the statistical models over more data)
• Appropriate pixel data formats (minimum bits per pixel, palettes instead of full colour, grayscale if appropriate). Automatically converting the spritesheets to all possible formats and picking the smallest files should be easy.