This is a difficult topic you won't find so much about on the net. We needed to do the same for our current project and wrote our own database in the end that fits our needs. What I can tell you is that you need some topics every database has and to develop some on your own if you intend to write a database.
First of all the file management, we decided to go for a single file database of a given size per chunk. Every chunk is an own type of table(s) we manage so if a chunk is full, we can move on to open a new table of the same type anywhere else in the file. Those tables are made for a single kind of data, for example:
-
Managing the type and address of tables
-
Managing the data itself
-
Managing indices of key/ value pairs
I decided for a model where those tables are chunks of data that can have child chunks addressed inside them and so go down up to the lowest byte range you need for a single field. A chunk in the file is always the root record, then we make smaller records from that root (minimum 4 for lagre tables, maximum 8 for smaller tables) and those records are also splitted into smaller parts (header and content of the table). On qeurry data on a tables content we addtionally split the data into a record that is defined by the table and addressed by the index. The size varies from a few bytes to 64byte chunks.
The search index is a BTree that keeps an undefined key type (for example a hash) and gives the coresponding database address of the data chunk we want.
Anything inside the database is thread-safe locked to either read or write access for performance reasons.
We use a transaction technik to keep track of changes. Changes to data will be cached in the transaction on commited on success only. This keeps the database clean from corrupted data when an errors occures that forces the transaction to cancel.
We also use some caching for any data that is requested from the database and every chunk accessed