I have been getting this really weird string crash.

Background:

I have a C++ function that is called by another C++ function, that is called from angelscript

inline void WStringToString(const std::wstring &ws, std::string &s)
{
    s = std::string(ws.begin(), ws.end());
}

void QuestData::SetQuestClass(std::wstring moduleName, std::wstring classname)
{
std::string modulenameAscii = WStringToString(moduleName);
std::string classnameAscii = WStringToString(classname);
}

Registered via:

    r = scriptEngine->RegisterObjectMethod("QuestData", "void SetQuestClass(string,string)", asMETHOD(QuestData,SetQuestClass), asCALL_THISCALL); assert( r >= 0 );

Called from angelscript via:
QuestData quest;
quest.SetQuestClass("Tutorial","TutorialData");

Using compiler MSVC2010, X86 target, debug mode

WStringToString crashes with: "Unhandled exception at 0x011dc989 in server.exe: 0xC000000F, Access violation reading 0xdddddde1"

Happens in xutility, on line: 146: _Mynextiter = _Parent_proxy->_Myfirstiter;

>	Server.exe!std::_Iterator_base12::_Adopt(const std::_Container_base12 * _Parent)  Line 146 + 0x6 bytes	C++
 	Server.exe!std::_Iterator_base12::operator=(const std::_Iterator_base12 & _Right)  Line 124	C++
 	Server.exe!std::_Iterator_base12::_Iterator_base12(const std::_Iterator_base12 & _Right)  Line 118	C++
 	Server.exe!std::_Iterator012<std::random_access_iterator_tag,wchar_t,int,wchar_t const *,wchar_t const &,std::_Iterator_base12>::_Iterator012<std::random_access_iterator_tag,wchar_t,int,wchar_t const *,wchar_t const &,std::_Iterator_base12>(const std::_Iterator012<std::random_access_iterator_tag,wchar_t,int,wchar_t const *,wchar_t const &,std::_Iterator_base12> & __that)  + 0x2f bytes	C++
 	Server.exe!std::_String_const_iterator<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >::_String_const_iterator<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >(const std::_String_const_iterator<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > & __that)  + 0x2f bytes	C++
 	Server.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::basic_string<char,std::char_traits<char>,std::allocator<char> ><std::_String_const_iterator<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > >(std::_String_const_iterator<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > _First, std::_String_const_iterator<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > _Last)  Line 623 + 0x37 bytes	C++
 	Server.exe!WStringToString(const std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > & ws)  Line 108 + 0x56 bytes	C++
 	Server.exe!QuestData::SetQuestClass(std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > moduleName, std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> > classname)  Line 107 + 0xd bytes	C++
 	Server.exe!CallThisCallFunction(const void * obj, const unsigned long * args, int paramSize, void (void)* func)  Line 1053	C++
 	Server.exe!CallSystemFunctionNative(asCContext * context, asCScriptFunction * descr, void * obj, unsigned long * args, void * retPointer, unsigned __int64 & __formal)  Line 170 + 0x1b bytes	C++
 	Server.exe!CallSystemFunction(int id, asCContext * context, void * objectPointer)  Line 486 + 0x37 bytes	C++
 	Server.exe!asCContext::ExecuteNext()  Line 2309 + 0x12 bytes	C++
 	Server.exe!asCContext::Execute()  Line 1144 + 0x8 bytes	C++
 	Server.exe!ScriptManager::ExecuteScript()  Line 234 + 0x17 bytes	C++
        <Snip, more of my own code>

Now the weirdest thing is, changing the C++ code to

void QuestData::SetQuestClass(std::wstring moduleName, std::wstring classname)
{
std::wstring tmp = moduleName;
std::string modulenameAscii = WStringToString(tmp);
tmp = classname;
std::string classnameAscii = WStringToString(tmp);
}

And it works fine. But I don't want to use that solution as it seems like this is an indication of a underlying bug.

I disabled MSVC's 'internal hiding' of std::wstring and what I found was this:

std::wstring moduleName Contains _String_val,

_String_val Contains _Container_base12,

_Container_base12 Contains _Myproxy

_Myproxy contains _Mycont. All valid values up to here.

But _MyCont then holds a _Myproxy value of 0xDDDDDDDD (Very similar to my access violation)

All other wstring's in my program contain valid values there, But for some reason once they leave angelscript that _Myproxy value gets messed up.

Tried it with stringpool enabled and disabled, no change in behavior.

Also, Here is my custom wstring code for angelscript (Well, Mainly its just a very simple port of the string code)

http://pastebin.com/Zx5kvMfe

Does WStringToString look like

inline void WStringToString(const std::wstring &ws, std::string &s)
{
   s = std::string(ws.begin(), ws.end());
}

or

inline std::string WStringToString(const std::wstring &ws)
{
   return std::string(ws.begin(), ws.end());
}

You use it as if it was the latter, yet you showed the first option.

Ah sorry.

Missed a line, Heres the actual function. (Its used elsewhere in the program and works fine, But it crashes ANY time I use it on a string past by copy from angelscript as a function pram, even in other places in my (rather complex) program)

inline std::string WStringToString(const std::wstring &ws)
{
	std::string s = std::string(ws.begin(), ws.end());
	return s;
}

The problem is with passing the string by value to application registered functions. When this is done AngelScript does a bitwise copy of the object to the stack in order to place the memory of object as the function expects it. Unfortunately Microsoft has in its "infinite wisdom" implemented a back-reference within the object to the object itself which doesn't get updated with the bitwise copy and thus points back to the memory location. It appears this back-reference is only there in debug mode, so it is obviously only used for debugging purposes.

Unfortunately this is not a problem that I can easily fix in AngelScript. Fortunately there is a quite easy work-around, and that is to change your functions to take the string by reference instead of by value.

void QuestData::SetQuestClass(const std::wstring &moduleName, const std::wstring &classname)
{
std::string modulenameAscii = WStringToString(moduleName);
std::string classnameAscii = WStringToString(classname);
}

r = scriptEngine->RegisterObjectMethod("QuestData", "void SetQuestClass(const string &in, const string &in)", asMETHOD(QuestData,SetQuestClass), asCALL_THISCALL); assert( r >= 0 );

Oh, I much prefer passing strings by ref anyway. For some reason I assumed that I could not pass strings by ref because they where a value type? I must of read the documentation wrong.

http://www.angelcode.com/angelscript/sdk/docs/manual/doc_register_type.html

"a value type doesn't support handles and can be passed by value to application registered functions."

Ahh, that is where I made the mistake. Maybe that should be updated to say "and can be passed by value or reference to the application registered functions"

Wouldn't the fix be using the (Script type provided) copy constructor instead of a bitwise copy when passed by value? Since that properly calls the std::wstring's copy constructor. I assumed that was how the copy was made since I saw 'CopyConstructStringGeneric' getting called. (maybe it was being called for some other code)

Changing it to pass by ref worked. Not sure why I didn't realize you could pass value types by ref to application functions, the string registration code does it all over (and has to, since it is registering the copy constructor itself)

PS: BMFont is awesome too. been using it to generate font sets for my games for ages and recommend it to anyone who asks about font rendering. I see it has really improved over the years!

I'll update the manual to be more clear about the use of references for value types :)

Yes, the fix is to use the copy constructor instead of bitwise copy. Unfortunately this has to be done in the low-level assembler routines used to set up the callstack and CPU registers, so it is not just a couple of lines of code that needs to be changed. It's not that I won't fix it, it's just that it will take a while and not the usual few days. I'll probably need to rewrite most of the code in as_callfunc_x86.cpp in order to avoid the bitwise copy.

PS. Thanks for the compliments on BMFont too. I'm glad you like it.

Ah. Ouch. I hate x64/x86 ASM, And I understand that changing ASM is not easy without large rewrites. Thankfully I can just avoid passing (non-pod) types by value now, Now that I know what the bug is. It just really scared me that string was somehow crashing and I needed to know it was not a symptom of something else I needed to fix in my code (Stack corruption or some such).

You might also think MSVC is silly with its debug mode strings.. But you do have to admit, It did manage to find this bug! And technically the copy constructor is the correct way, even if it might end up being a little slower and would only be noticed in 0.1% of cases.

I am happy I could find a bug and help you improve Angelscript. It appears to also have had a ton of improvements since I first tried it (2.24). Oh that is right. Modfusion (http://newvegas.nexusmods.com/mods/46859) also used Angelscript. I used it for the install scripts (Detects other installed mods, DLC, etc) and to actually alter the mods themselves, as my mod manager actually parsed the entire mod like fallout new vegas would, and then would use Angelscript to call C++ functions to alter the data before writing it back to disk. Thought you might like to hear where Angelscript has turned up! Made installing of some of the other mods I wrote a LOT easier, and made some crazy mod ideas of mine possible, Like scripted replacement of objects in the game (Turning 9000 rocks and bushes into random zombies was fun..)

Also, Feel free to add that wstring version of your string registration function to Angelscript (http://pastebin.com/Zx5kvMfe , based on 2.24's string registration function). between that and setting unicode support in scripts, Angelscript has worked well in a unicode environment... Well, Except for the fact that all your C++ side functions still only take char* strings for function defs and such, so I have to run that WStringToString function on all the strings first before using them in angelscript. But ah well, Angelscript is not the only library I have to do that for and people can live with ASCII only function names if at least the strings they show users can be in unicode.

I don't think MSVC is silly (though I truly was being sarcastic when writing 'infinite wisdom'). Their debug libraries have helped me a lot in the past, and still do. I'm all for debug code, and I also have a lot of it within AngelScript to catch errors that are otherwise hard to track down.

It is a bit unfortunate that the debug code in MSVC's string, prevents the memcpy/memmove of the object, as there is otherwise no other reason why you shouldn't be able to move the memory as long as you know that no outside references to the object is still looking at the old address. But I can definitely see that other types of objects have a need for this back-reference, so the problem would likely have cropped up eventually anyway.

Thanks for letting me know about Modfusion. I've added it to the user list now.

I am really glad for the debug code in angelscript. It is so nice getting decent error messages in my logs that point at what is wrong.

Yea. I don't really have any idea what that proxy/cont stuff is about. Reading MSVC's STL is like torture. I am just glad they work. And yea, eventually someone ends up calling foo->bar and bar calls its own pointer to foo and all hell brakes loose. I think I already have cases like that in my program (But passed by ref/handle everywhere).

Cool. I wish modfusion became more successful for fallout modding but it seemed other authors didn't pick it up, even after I showed them how simple it was to write install scripts. Oh well, it installs my mods great and massively reduced complaints due to incorrectly installed mods. And it let me do some nifty things. Ah well, 10,000+ downloads is still very nice.