Hello! I have started using AS and it's been a blast. The best scripting language I have used hands down.

But now I have hit a hurdle. I'm making a block based system - you take many blocks and link them all together. Every block is programmed in AS and threaded. Now the question is which parts I can thread? The doc's says the engine is totally thread safe if I only use one. Couldn't see anything specific about contexts or modules. Recent commits seems to have put locks on modules though (haven't tested the WIP version), which seems to me that it wasn't thread safe. I have tried this:

//This is done once
asPrepareMultithread();
asIScriptEngine *engine = asCreateScriptEngine(ANGELSCRIPT_VERSION);

//Then in each thread:
asIScriptContext *ctx = engine->CreateContext();

string code;
as::LoadScriptFile(script, code);

asAcquireExclusiveLock();
asIScriptModule *mod = engine->GetModule("my_module", asGM_ALWAYS_CREATE);
asReleaseExclusiveLock();

mod->AddScriptSection("my_module", code.c_str());
// Build the module
int r = mod->Build();
if( r < 0 )
{
  printf("Please correct the errors in the script and try again.\n");
  return;
}

asIScriptFunction *func = mod->GetFunctionByDecl("void main()");
if( func == 0 )
{
  printf("The script must have the function %s. Please add it and try again.\n", function.c_str());
  return -1;
}

while (true){ //This executes the function in the thread until it breaks
  ctx->Prepare(func);
  int r = ctx->Execute();
  if (thread_break == true) break;
}

ctx->Release();
asThreadCleanup();

//Here threads end

The problem here is that it sometimes crashes. When I put asAcquireExclusiveLock(); around every as call then it doesn't, but then it impacts the framerate as it stalls for large amount of time. So what else needs to be locked specify? I could experiment and find out, but I would like to know what is recommend. Like if GetModule are not thread safe, then is GetFunctionByDecl neither? What is thread specific? Like can I call a function from the same context in two threads the same time (if they don't have any shared or global variables)? Do I need to Release "function" pointers just like I need to release contexts and engines?

And I seem to have problems with "Script section empty" error. I have tried several methods on loading files and I don't know why, but it sometimes fails. I tried the ScriptBuilder, as well as code from "Compiling scripts" section of the docs. When I have this error, I have to modify and save the file again for it to load. I do autoload files though based on their modify date, which could be the issue (I load it before it has finished writing on the disk??). I will add a delay and see if that helps. I do see that in threads formatInt() and other string functions crashes. Not sure if I'm at fault.

Another question is about variable sharing.

I need to have several blocks, like "BlockA" and "BlockB". I need have several instances of the same block, like "BlockA1" and "BlockA2". Then I need to execute the blocks in parallel without them interfering with one another. In AS terms each of my unique blocks (A and B) would be separate modules - "ModuleA" and "ModuleB". Then separate instances of blocks would be different contexts "ContextA1" and "ContextA2" right? So if contexts are different, then they won't interfere with one another? Like I need them to have global variables (declared outside any function, so it holds the state if you will), because each block is made out of several functions. Like this:

int s = 0; //State that doesn't change between function calls

void funcInit(){ //This is called from C++
 s = 5;
}

void funcUpdate(){
 s = s + 1; //This changes the state and is called from C++
}

I guess this is how something like that would work?

The GetModule() call was not thread-safe until recently. This has been corrected in the WIP.

The rest of the calls that you make in the code you share should be perfectly safe. You should however be aware that the Build() call will fail with return code asBUILD_IN_PROGRESS if another thread happen to be compiling a script at the moment, so you'll need to add logic to wait and retry if you don't want your code to fail completely in this case. I recommend creating a queue for build requests so all the compilation is always done from the same thread and the other threads only requests builds from it. I believe that would be easier to manage (though I'm far from an expert on designing multi-threaded applications).

modules do not interact with each other unless you have explicitly implemented some code to allow this interaction through the application. So if only one thread accesses the same module at any time this should be perfectly safe.

The same for contexts, each context has its own internal state so unless you have two threads accessing the same context at the same time you shouldn't have any trouble.

You don't need to 'release' function pointers received from GetFunctionByDecl(). I specifically mention in the manual the pointers that needs to be released, but as a general rule any function named 'Get' will return the pointer without increasing the reference counter, and any function named 'Create' will return a pointer that has to be released afterwards.

formatInt (from scriptstdstring.cpp add-on) should be thread safe as long as you make sure to link with the multi-threaded version of the C-runtime library.

The memory for the global variables are stored in the modules. If two contexts call functions on the same module they will both interact with same global variables in that module (and will thus not be thread-safe). If two instances of the same Block share the same script module they will also share the same global variables, even if they do not share the same context. If you use script classes and store the state in the class members rather than global variables, then you could create separate instances of the classes for each block, thus allow multiple blocks to use the same module yet avoid conflict. Should you go with this, I also recommend disabling the use of global variables in the script with SetEngineProperty(asEP_DISALLOW_GLOBAL_VARS, true) so you don't accidentally have any scripts use global variables without your control.

There were a lot of questions in your post. Hopefully I answered them all, if not, please ask again.

Please do let me know if you encounter any problems with multi-threading. I honestly don't have the means to guarantee that the implementation is perfectly thread-safe in all situations, but I always strive to improve the library and for that I need the feedback from the community.

Thank you for all the answers.

You should however be aware that the Build() call will fail with return code asBUILD_IN_PROGRESS if another thread happen to be compiling a script at the moment, so you'll need to add logic to wait and retry if you don't want your code to fail completely in this case.

This might be the issue. I ran Build on several threads thinking it will just wait internally. I will probably do your queue method later, but right now I just did while (r == asBUILD_IN_PROGRESS) r = mod->Build(). For now I haven't gotten the empty section error.

You don't need to 'release' function pointers received from GetFunctionByDecl(). I specifically mention in the manual the pointers that needs to be released, but as a general rule any function named 'Get' will return the pointer without increasing the reference counter, and any function named 'Create' will return a pointer that has to be released afterwards.

Good tip! It's just that GetModule() also creates the module (with proper flag or when using scriptbuilder) and so I was just thinking GetFunction also does that. Only when function->Released() crashed my program did I came to realize it's somehow different.

formatInt (from scriptstdstring.cpp add-on) should be thread safe as long as you make sure to link with the multi-threaded version of the C-runtime library.

I compile with GCC and -pthread flag (which links with -lpthread and defines the -D_REENTRANT. I also tried disabling string pool thinking it might not be thread safe, but after looking at the formatInt function I don't see why it would crash for me. It just does sprintf.

So I went back to no-threading just to see if the problem is somewhere else. And weirdly enough it now crashes without threads too. It did work previously though, so I'm not sure what's the problem. Will look into this in the morning.

When I fix my integer to string problem I will look into how to deal with globals painlessly.

edit: 30 seconds after posting I tried another thing which I totally forgotten - I use Jit by BlindMindStudios. And I haven't tried formatInt with Jit. It turns out that it is the reason why it crashes. If I remove Jit the function works fine. Can anyone replicate this?

edit: 30 seconds after posting I tried another thing which I totally forgotten - I use Jit by BlindMindStudios. And I haven't tried formatInt with Jit. It turns out that it is the reason why it crashes. If I remove Jit the function works fine. Can anyone replicate this?

You should probably post about this on BlindMind's git hub for the JIT compiler. While ThyReaper do show up here once in a while you'll have a greater chance of getting a response on potential problems with the JIT compiler over there.

I have weird results with globals. The docs doesn't say how the globals a defined, it just says:

Global variables may be declared in the scripts, which will then be shared between all contexts accessing the script module.

Variables declared globally like this are accessible from all functions."

Which is okay, as I understand "global" as in C++ global. That means in script like this:

int a = 0; //This is global
void main(){
  print("A = "+formatInt(a,"")+"\n");
  ++a;
}

the int a is global right? I get confirmation on my theory when I set "SetEngineProperty(asEP_DISALLOW_GLOBAL_VARS, true)" which makes the script error out because of the global variable. But if I load the same code in two theads with two contexts (but the same module name) I get this output:

A = 0
A = 1
A = 2
A = 3
A = 4
A = 5
A = 6
A = 7
A = 8
A = 9
A = 0
A = 10
A = 11
A = 12
A = 13
A = 14
A = 15
A = 16
A = 17
A = 18
A = 19
A = 20
A = 1
A = 21
A = 22
A = 23
A = 24
A = 25
A = 26
A = 27
A = 28
A = 29
A = 30
A = 31
A = 2
A = 32
A = 33
A = 34
A = 35
A = 36
A = 37
A = 38
A = 39
A = 40
A = 41
A = 42
A = 3

One thread sleeps for 100ms after each script call, while the other sleeps 1100ms. I would expect to see a getting incremented all the time, but here the two threads use a different variable "a". If a is global, then it should be shared by contexts within a module no?

This is actually what I need and what I want, but it contradicts the docs. Maybe I'm doing something wrong.

And how would I create a unqiue struct instance inside AS? I want to make a system where a person can create all the things he needs only using scripts, so he wouldn't need to create a C++ struct and pass it in. So I get that I can do this:

struct number{
  int a;
}

void init(){
  number n;
  n.a = 5;
}

void update(){
  //How to access unique number.a? I could access global a here.
}

But each execution will have the same object?

Are you sure both threads are actually using the same module? Check the pointer of the asIScriptModule that you get from GetModule. Based on the code you shared previous, you create a new module each time you load a script (asGM_ALWAYS_CREATE), thus your code doesn't actually share the same module.

Observe that since you use the same name in both calls to GetModule, the first module instance is discarded when the new one is created. Discarding a module that is still in use will not immediately destroy it, so the first thread is still seeing and incrementing the global variable in the first instance, while the second thread sees and updates the second instance.

On the option of using script classes I suggest you encapsulate all the logic and memory in the controlling script class. The C++ class would create the script class instance, and then call the update method of that class instead of calling a global function.

class Controller
{
  int a;
  void init() {
    a = 5;
  }
  void update() {
    print("A = "+formatInt(a,"")+"\n"); 
    ++a;
  }
}

The application can create the controller class like this:

// Create the instance of the script class
asIObjectType *type = mod->GetObjectTypeByName("Controller");
asIScriptObject *obj = reinterpret_cast<asIScriptObject*>(engine->CreateScriptObject(type));
 
// Call methods on the script class
asIScriptContext *ctx = engine->CreateContext();
ctx->Prepare(type->GetMethodByName("update"));
ctx->SetObject(obj);
ctx->Execute();
ctx->Release();
 
// When you're done, don't forget to release the script object
obj->Release();

(of course, the code above doesn't have appropriate error handling. ;))

Read the following article in the manual for a more detailed explanation: Using script classes