Thanks. It's good to know the solution worked well. I have a little performance test that auto generates a script with ~40 thousand lines, and then compiles it. With the optimizations the compilation time improved almost 4 times, but the autogenerated script doesn't really reflect a real world script, so I wasn't too sure the improvements would be that good.

I'll see if I get the time to look into improving the GetFunctionDescriptions. 30% of the time is quite a bit, and it shouldn't be too difficult to replace the current linear searches by storing the functions in a map, similar to the enhancement done for the string constants by vroad.

The asCompareStrings() function is more difficult to improve. It would be easier to eliminate the string comparisons from the calling functions. Can you see where it is called the most? GetFunctionDescriptions() would be one one of them, but where else?

As an additional, slightly dumb trick, have you investigated using something like Boost's flat_map structure? Since the number of tokens can be easily predicted and isn't likely to change at all once the lookup table is constructed, this could be an interesting (and fairly easy) way to avoid cache misses. For the uninitiated, it's basically a vector-based map structure that sorts the contents in a method similar to how std::map works instead of using a linked list structure to hold entries.

Thanks for the tip. I'll have a look at boost's flat_map structure to see if it something I can easily use.