Advertisement

Wrong function called on bytecode restoration

Started by February 09, 2013 07:12 PM
23 comments, last by WitchLord 11 years, 9 months ago

Tested on rev1549. The bug is sporadic and difficult to pinpoint, its root cause is likely in the bytecode generation rather than bytecode restoration. In a script with a registerd value-type variable placed in the global scope, sometimes another function is called instead of the variable's type constructor when the script is loaded from the bytecode.

Loading the script from the source is without problems. When loading from the generated bytecode, it either works correctly all of the time (for that generated bytecode), or it consistently calls the same, wrong function (which has no special relation to the correct one). Whether the bytecode is buggy or not seems to depend on things like order of compilation of scripts or whether some script was compiled prior to the one in question.

Calling the wrong function naturally leads to bad behaviour, here is the call stack I get when I stumble upon a resulting crash:


    CallCDeclFunctionObjFirst + 38, as_callfunc_x86.cpp (494)
    CallSystemFunctionNative + 885, as_callfunc_x86.cpp (207)
    CallSystemFunction + 294, as_callfunc.cpp (486)
    asCContext::ExecuteNext + 2756, as_context.cpp (2450)
    asCContext::Execute + 521, as_context.cpp (1158)
    asCModule::CallInit + 337, as_module.cpp (310)
    asCModule::ResetGlobalVars + 31, as_module.cpp (254)
    asCReader::Read + 278, as_restore.cpp (98)
    asCModule::LoadByteCode + 111, as_module.cpp (1189)

Maybe it is important that there could be some functions etc. that are registered to the engine after some scripts are already compiled (or loaded from the bytecode). This never caused any problems before though. For the record, the value-type in question is registered this way:


    if(ASEngine->RegisterObjectType("FuncBind", sizeof(ScriptFuncBind), asOBJ_VALUE) < 0)
        Log("%s FuncBind\n", fail);
    if(ASEngine->RegisterObjectBehaviour("FuncBind", asBEHAVE_CONSTRUCT, "void f()", asFUNCTION(ScriptFuncBind_Construct), asCALL_CDECL_OBJFIRST) < 0)
        Log("%s FuncBind ctor\n", fail);
    if(ASEngine->RegisterObjectBehaviour("FuncBind", asBEHAVE_DESTRUCT, "void f()", asFUNCTION(ScriptFuncBind_Destruct), asCALL_CDECL_OBJFIRST) < 0)
        Log("%s FuncBind dtor\n", fail);

This sounds like it may be the same problem reported here:

http://www.gamedev.net/topic/637417-losing-some-function-pointers-after-bytecode-reload/

Unfortunately I have not been able to reproduce the problem, much less figure out what causes it.

I'll see if the information you provided perhaps allow me to find something.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Advertisement

There is yet another symptom of probably the same bug: after loading all scripts except the last one from the bytecode, and then the last one from the source, it did not compile and produced an interesting error message:


Info : Compiling void ask(Critter&inout, int, int, int) : 8, 1.
Error : No matching signatures to 'Is()' : 10, 5.
Info : Candidates are: : 10, 5.
Info : bool Flee(Critter&inout, bool) : 10, 5.


After some debugging, it turns out that the contents of asCScriptEngine::scriptFunctions apparently consisted of a total of 72852 entries, some 3/4 of which were nulls (I assume some padding, maybe removal of duplicates, in any case many array indices and function ids were unused). In particular, there were functions with index (= id) over 65535. The problem is that asCBuilder::GetFunctionDescription treats them as imported (it checks for the 0xFFFF0000 flag, even though I see that the imported functions have ids beginning from 0x40000001) and searches for them in asCScriptEngine::importedFunctions rather than asCScriptEngine::scriptFunctions, giving a match to some random function that has been imported somewhere rather than the right function.

Seems that the problem can only manifest itself in codebases large enough. Apparently we hit this hidden function limit some time ago. By the way, what is the reason for this index/id skipping in asCScriptEngine::scriptFunctions?

That shouldn't happen. The empty slots in scriptFunctions should be reused.

This may very well be the cause of trouble. I imagine you use shared functions and classes, am I right?

I'll investigate why the function ids aren't reused as they should.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

We do use shared classes. I notice that functions from a single module and functions that are specializations of some registered global function/methods that use classes from that module form uninterrupted blocks in scriptFunctions, as if null gaps are added after a module has been entirely loaded from the bytecode.

What about that possible inconsistency with GetFunctionDescription using 0xFFFF0000 flag to determine that the id denotes an imported function, where in reality their ids start from 0x40000001 (or so it seems)? We use ~17000 unique ids for non-imported ones at the moment. That's not 65535 yet, but it's the same order of magnitude.

Yes, the 0xFFFF0000 mask is likely why you're seeing the wrong functions being called. I will change this mask too, however, this is just an indirect cause because with only 17000 unique ids you shouldn't be reaching the 65535 limit.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Advertisement

I believe I found the problem. I've checked in the fix in revision 1555.

This should solve the problem with the empty slots in scriptFunctions not being reused as they should. I haven't changed the mask for imported functions yet, but if I'm right your problem will already be solved with the changes I made.

Please let me know if this corrects your problem, and if you still see a lot of unused slots in scriptFunctions.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

I still see those gaps. I made a stack trace whenever asCScriptEngine::GetNextScriptFunctionId() returns id that is eventually a null entry in scriptFunctions, for each of ~100 functions I checked it's always the same:


asCScriptEngine::GetNextScriptFunctionId()  Line 4409    C++
asCReader::ReadFunction(bool & isNew=true, bool addToModule=false, bool addToEngine=false, bool addToGC=true)  Line 646 + 0xb bytes    C++
asCReader::ReadInner()  Line 350 + 0x12 bytes    C++
asCReader::Read(bool * wasDebugInfoStripped=0x00000000)  Line 71 + 0x8 bytes    C++
asCModule::LoadByteCode(asIBinaryStream * in=0x0d533f0c, bool * wasDebugInfoStripped=0x00000000)  Line 1187 + 0xf bytes    C++

Trying to step debug to the end of ReadFunction didn't reveal much, but I can tell that the function was imported.

[edit] A wild guess: imported but unused functions?

OK. I'll continue to investigate. Apparently there are more places that causes gaps in the scriptFunctions.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

as_restore.cpp:350   info->importedFunctionSignature = ReadFunction(isNew, false, false);

ReadFunction internally requests an id to be given, even though info looks to be an information about an imported function, which gets its own, high id anyway. Isn't that a problem?

This topic is closed to new replies.

Advertisement