Compressing LDraw database

posted in DevJournal for project QLMesh 2.0
Published July 14, 2018
Advertisement

One of the main goal for QLMesh was to add some new formats I have been working with quite often, like Photoshop files of bdf fonts.

For 3D it is LDraw formats and DAZ Studio models.

LDraw is one of my favourite. I am currently working on extending Assimp to support .ldr and .mpd files. One of the major challenge is actually not drawing but embedding library definitions into the plugin. Original library it is about 250MB (compressed to ~40MB). That's quite large for Quicklook plugin. I started to work on some heavy compression/optimalization and current result is:


 

-rw-r--r--  1 piecuchp  staff    40M May 12 17:18 parts.db
-rw-r--r--  1 piecuchp  staff   2.2M May 12 17:18 parts.db.gz

That's much better. 2MB can be easily embedded into plugin, eg. using assembler module like this:


 

bits 64

section .rodata

global _ldrawlib
global _ldrawlib_end
global _ldrawlib_size

_ldrawlib:      incbin "parts.db.gz"
_ldrawlib_end:
_ldrawlib_size: dd $-_ldrawlib

and later build with e.g. nasm: 


/opt/local/bin/nasm -fmacho64 ldraw_lib.asm -o ldraw_lib.o

 

PS1

Sometimes less is more. Working on reading gzip stream, I had to remove one of the compression optimisation. The uncompressed file is slightly bigger, but compressed one much smaller:


-rw-r--r--  1 piecuchp  staff    41M Jun 17 12:03 parts.db
-rw-r--r--  1 piecuchp  staff   1.5M Jun 17 12:03 parts.db.gz


PS2

Sadly, this is not the end of the story :) I had to increase the precision of the float numbers in the database (it is now 17 bits - sign:8bit:8bit) - it increased the size but also significantly affected the compression ratio:


-rw-r--r--  1 piecuchp  staff    67M Jul 11 08:55 parts.db
-rw-r--r--  1 piecuchp  staff    41M Jul 11 08:55 parts.db.gz

Seems like I am gonna have to live with such database for a while.

Screen Shot.jpg

Mech2.jpg

0 likes 0 comments

Comments

Nobody has left a comment. You can be the first!
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Profile
Author
Advertisement
Advertisement