Hi guys, I'm having an issue with loading async files from disk.

Let me give some context, code is in C++, I have 2 threads, main thread ( physic & render ) & IO thread ( in charge of loading files async ). Sometimes a request, while being loaded in the IO thread ( I'm using Windows API with ReadFileEx and SleepEx ), is removed from MainThread, causes the callback to fail. ( for example I moved too fast, and the loading request didn't finish, I was loading a texture, but the object is too far away from me, so I need to delete it )

I know one option is to not delete the request until read is done, but I don't like this approach because I don't like keeping objects with a pending request in a Update loop only checking if it was completed ( because I will need to implement that kinda Update check system for each class I create ).

Another idea is to create a list of loaded files, which will be called with the render fn from MainThread. The list contains all the files already read from the IO thread, if the request was deleted I just simply delete the memory, if the request is still alive, I do the callback.

The issue I saw in here is, what happen if I want to create a new thread, and I need to load any file from that new thread ( can be a task thread ), I would need to create a new list for that thread, and a function for it too.

Any ideas, lib or code that I can check about this?

So to let me summarize that, you have trouble with resource loading?

If so then let me introduce you to the wonderful world of resource/asset managing. Usually if you require something to exist, a mesh, material, shader, texture but also some level geometry you have two options:

1. Load everything on the same thread which requested it (slow, game starts to stutter)
2. Dedicate everything to an authority which takes care of such resource requests

You don't want to take the first one so we go for the second one. Those authorities are usually resource/asset manager classes. Those classes act on another thread (so you are async anyways) and take care of let's say priority but also lifetime of the resource requested. (There is btw. a good post about resource loading some threads below this one). What it does is locating the resource on the disk, usually it is placed along with other resources in a package which is loaded in chunks into memory to access the resources required. Loading a resource is never suspended at certain point and it is never exclusive. I guess this is the issue you are running into in your design, exclusive resource access which is strictly coupled to a single instance. Instead a cache is maintained which keeps track of loaded resources and their lifetime. If the cache runs to a limit, it executes a cleanup which removes unused assets with the highest lifetime from memory for as long as there are still unused assets present or the memory limit is reached where it can stop.

The problem you might run into sooner or later is: what happens if you have a resource used on a single object in the game only and it is swapping in and out of sight frequently? Do you always trie to erase the memory and load the entire file again from disk, of course you won't! Else your game is considered to bug and break on 50% of all players because people are not acting like you expect them to.

So instead of loading an asset for single objects, you have to load them into a cache and pass a handle like struct to the caller which you maintain in the resource manager. That handle usually points to a default asset (like the puprle textures you see when a load fails) and is replaced immediately after the resource has finished loading.

If this isn't the case and you have a different issue, please explain in more detail how you expect it to work

Object will be handled in routine as not visible etc so best aproach in my opinion would be delaying deleting of object, and have next deleting seeking routine handle deleting of completely loaded object. This is closer to memory managed language behavior.

Shaarigan said:

So to let me summarize that, you have trouble with resource loading?

If so then let me introduce you to the wonderful world of resource/asset managing. Usually if you require something to exist, a mesh, material, shader, texture but also some level geometry you have two options:

1. Load everything on the same thread which requested it (slow, game starts to stutter)
2. Dedicate everything to an authority which takes care of such resource requests

You don't want to take the first one so we go for the second one. Those authorities are usually resource/asset manager classes. Those classes act on another thread (so you are async anyways) and take care of let's say priority but also lifetime of the resource requested. (There is btw. a good post about resource loading some threads below this one). What it does is locating the resource on the disk, usually it is placed along with other resources in a package which is loaded in chunks into memory to access the resources required. Loading a resource is never suspended at certain point and it is never exclusive. I guess this is the issue you are running into in your design, exclusive resource access which is strictly coupled to a single instance. Instead a cache is maintained which keeps track of loaded resources and their lifetime. If the cache runs to a limit, it executes a cleanup which removes unused assets with the highest lifetime from memory for as long as there are still unused assets present or the memory limit is reached where it can stop.

The problem you might run into sooner or later is: what happens if you have a resource used on a single object in the game only and it is swapping in and out of sight frequently? Do you always trie to erase the memory and load the entire file again from disk, of course you won't! Else your game is considered to bug and break on 50% of all players because people are not acting like you expect them to.

So instead of loading an asset for single objects, you have to load them into a cache and pass a handle like struct to the caller which you maintain in the resource manager. That handle usually points to a default asset (like the puprle textures you see when a load fails) and is replaced immediately after the resource has finished loading.

If this isn't the case and you have a different issue, please explain in more detail how you expect it to work

Hmm, interesting about the resource with a cache, I will look at it.

Problem is, the engine is a little bit old and I don't have the whole source of it, it's a single-thread designed ( or looks like that ), so I need to check if it is possible to load from an async thread meshes, material, textures.

Right now my approach is to first read files from disk in another thread ( because can take some time ), allocate memory for it, and add it to a list when it's done, and in the main thread, before render, I just iterate through that list, get the bytes read, put them into the stream and for example creates texture from stream, that way, if a file is too big, it willn't impact in the game FPS, because was load in another thread.

The engine have the texture class, it's like a shared pointer, u create the texture the first time, then it will create a counter with all the references it has, when the counter go back to 0, then it will destroy the texture, but yeah, if only a single object of the game use one texture, and it's swapping in and out of sight frequently, I always destroy it, read memory from disk and load it again, prob I need to do some cache system for it.

Also, I use the async IO thread to read files, one good example is the map sectors, I have multiple files sector0 ~ 10 from map 1, the code before readed those sectors and started to load the sector, if the file was too big that takes 300ms to read ( I'm using it as an example ) the game will freeze for those 300ms until the sector was read from disk.

What I do now, is send a request to the async IO thread with a callback, so if the file takes 300ms to read from disk, I don't care, that will be done inside another thread, and when bytes are read, I just call the callback, copy the memory and set flag “read = true”, now when main thread updates the loading sector system, it will check if ( read == true ) { StartLoadingSector(); }, my issue is, what happen if I left the sector, while it was being read? The async IO doesn't know the file isn't needed anymore, so it keeps reading it, when read is done, it try's to do the callback to a deleted pointer.

That's my issue with every file request I do, if I don't need the file anymore, how to avoid doing the callback to a null object ( because objects are deleted in the main thread, while async IO is reading files )

YJohnnyCode said:

Object will be handled in routine as not visible etc so best aproach in my opinion would be delaying deleting of object, and have next deleting seeking routine handle deleting of completely loaded object. This is closer to memory managed language behavior.

I'm thinking on doing something like this, but with the cache system.

Rewaz said:
but with the cache system.

Even better really.

Rewaz said:
Also, I use the async IO thread to read files, one good example is the map sectors, I have multiple files sector0 ~ 10 from map 1, the code before readed those sectors and started to load the sector, if the file was too big that takes 300ms to read ( I'm using it as an example ) the game will freeze for those 300ms until the sector was read from disk. What I do now, is send a request to the async IO thread with a callback

What engine do you use and is this really async I/O as in Windows OVERLAPPED I/O or pseudo async handled from the engine itself? If you have control over that system, consider to use memory mapped files instead. Memory mapped files utilize the virtual memory manager to handle read/write requests onto a phsical piece of HDD space for you. It is widely used in AAA gaming to provide massive performance improvements related to disk I/O.

You then usually place everything into a single, very large file on disk which may even exceed the RAM limit of 64 GB easily. The trick is that OS is handling the requests in a way that it only loads those regions you are currently interested in into memory pages and swaps those pages out that aren't used anymore. In my previous job we used that technique to load a database of up to 5 TB into our editor tool without ever hitting the 2 GB RAM limit.

Also, those memory mapped portions of the file are thread-safe by default because it is just a pointer to a virtual memory region in RAM. Multiple threads can read them at the same time without causing any I/O related delays

It's the engine from GTA SA.

I use PAK system, where u have PAK header, and PAK body, and each body part have 100MB max. Example, I have texture.pak which contains all the file names, size of the file, offset and in which file it is, and then I have texture0.pak, texture1.pak, etc.

If I want to load “big_blue.png”, I find it inside the map from the PAK header, and it will tell me that I need to read texture2.pak, the size is 230450 and offset is 10325, and I read the file with those options. Then in the IO thread, I have a function called LoadFile, which do something like this

std::size_t BLOCK_SIZE = 64 * 1024;

// etc...
// determine block size
if ( blockCount > 0 )
{
	size = BLOCK_SIZE;
	blockCount--;
}
else
{
	size = remain;
	remain = 0;
}

ReadFileEx( hdl, (LPVOID)basePtr, size, &ov, BlockComplete );

void BlockComplete( DWORD dwErrorCode, DWORD dwNumberOfBytesTransferred, LPOVERLAPPED lpOverlapped )
{
	blockCount += dwNumberOfBytesTransferred;

	// check for remaining blocks
	if ( blockCount > 0 || remain > 0 )
	{
		ReadNextBlock();
	}
	else
		asyncDone = true;
}

//
while ( asyncDone == false )
{
	// allow our thread to enter an alertable state so the async callback function will be called
	SleepEx( 0, TRUE );
}

if ( callback )
{
	callback( instance, data, size );
}

I don't know if you do anything special with “asyncDone”, but if not, simple raw access to a variable is typically not a reliable way to communicate between threads.

You need to tell the compiler that ‘asyncDone == false' cannot be optimized away, and you need to tell the processor performing the assignment to synchronize memory with the processor sleeping.

asyncDone is only used in async IO thread, so it's safe to use, or I'm losing something? ReadFileEx creates another thread to read the file, or use the same async IO thread that I created?

The code is in here https://www.gamedev.net/forums/topic/382135-overlapped-file-io/3526051/