Hello!  This is the first of a few posts I have concerning AngelCode, which has truly been an angel and made it possible to pursue my open source project (more info about that in another post).  While looking at cscriptarray (version 2.33.0), I noticed it didn't check for a new pointer after a malloc in Precache(), around line 1694.  The fix I added (though I don't think I've tested it specifically yet) is:

// Create the cache
cache = reinterpret_cast<SArrayCache*>(userAlloc(sizeof(SArrayCache)));

/// BEGIN malloc check
if (! cache)
{
   asIScriptContext *ctx = asGetActiveContext();
   if( ctx )
      ctx->SetException("Out of memory");
   return;
}
/// END malloc check

memset(cache, 0, sizeof(SArrayCache));

Just wanted to note it as a potential bug for fixing in later versions.

Related, I have a more general question about how AngelCode handles malloc fails.  A quick inspection of the code suggests that most areas of the interpreter code cannot handle a malloc call failing.  Is this true or did I misread it?  Is this something under consideration for fixing in the future?

Basically, I'm using AngelScript in what is essentially a semi-open sandbox environment, where most users could (if desired) write code and have it execute on the server, often simultaneously with other scripts.  Because of this, I have to keep careful tabs on resource usage (CPU and memory).  For now, I'm checking memory used by each AngelScript engine via the debug hook (and a custom allocator) and aborting in there if it went over.  It would be much nicer to be able to abort during the actual allocation and clean up, but I suspect it would be way too difficult for AngelScript to check every single malloc.  Still, I thought I'd check!  Thanks for making such a nice, easy-to-integrate product!

Thanks for letting me know of the missing check for failed malloc. I'll make sure to add this fix for the next release.

The solution you implemented is the correct way to do it. It should be done the same way in every other memory allocation as well. You should be able to simply return zero on the custom memory allocator when you don't want to allow an engine to consume more memory. It should be treated by Angelscript.

Let me know if you find other places where the code doesn't treat failed mallocs.

Regards,

Andreas

You're welcome!   Also, what about the AngelScript interpreter itself?  Is it supposed to be able to handle a failed malloc internally at any time?  My guess is 'no' based on what I saw, but I did not do an extensive code analysis and the failure may have been caught further up the chain.  It was also an older version.

I'm thinking my periodic heap check every few times the debug hook is called should be sufficient, but the finer grained I can make it the better.

The code should be able to handle it, though I can't say I've tested every possible situation where a memory allocation might fail.

Just report any situation where you feel the code might not handle it properly and I'll fix it.

I've fixed the missing check for failed memory allocation in revision 2586.