I've found a solution in AngelScript. You can get it in revision 1134.

As the script function is still kept alive by the application when the engine is released, the engine will report this with an error message, and will destroy the internals of the script function. It won't destroy the actual function object, so the application can still call the Release() method afterwards without crashing.

It is not correct to release the engine before all other objects that use it has been released. I've tried very hard to avoid memory leaks should this happen, but I cannot guarantee that I've covered all possible scenarios.

I'm not sure what would be the best way to safely release all objects and also let the AS module about that. How do I do that?

Just releasing the objects that the application no longer will use should be enough. If AS is not currently referencing the object, then the object will be destroyed, otherwise it will be destroyed when AS releases its reference too.

As each reference counted object is responsible for destroying itself when the refCount reaches 0, the modules that references the objects doesn't need to know about whom might still be referencing the object.

The problem arises when objects may be referencing each other in a circular manner, preventing the refCount from ever reaching 0. That is why AngelScript has the garbage collector, to detect and destroy such occurances.

Perhaps you could explain a bit more about how your application is structured? So I can understand why it is not safe to simply release the objects before releasing the engine.

In our game AS is the only module that uses reference counting on objects. Basically all objects are stored in a huge preallocated array which is then freed at shutdown. Obviously we can't free that array prior to shutting down the AS engine otherwise it'll just crash upon referencing a freed memory region. Manipulating the reference counters before releasing the AS engine might be a possibility but that doesn't sound too safe.

In your test case you didn't release the test object in the application either and this is why the problem could be reproduced. How'd you modify the test case to avoid hitting the problem?

In my test I would simply release the 't' variable before I release the engine.

In your case it would probably make more sense to just release the callbacks, before releasing the engine. The actual objects that are in the preallocated array can be kept alive.

Using my test as example, the cleanup would look like this:


if( t->callback )
{
// Release the callback and any other objects obtained from the script engine before releasing the engine
t->callback->Release();
t->callback = 0;
}

// Release the engine
engine->Release();

// Release the application owned object last
t->Release();

However, if it is too much work then you should be able to leave your code as it is now with the fix I made in the library. The engine will complain about the callbacks not being released, but it shouldn't cause any problems for you. Even memory leaks at this moment probably isn't too much to worry about since the application is shutting down anyway, right?

Since our game library is written in C, I'll have to add ReleaseFunction (void *) method to the API and use that instead of t->callback->Release(). I'd prefer to have a clean shutdown procedure there.
As for the memory leaks, we're using memory pools in the game so any potentially leaked memory is going to be freed along with the parent pool anyway.