The following code checks for a one-line comment.

as_tokenizer.cpp(213):

if( source[1] == '/' )
{
    // One-line comment

    // Find the length
    size_t n;
    for( n = 2; n < sourceLength; n++ )
    {
        if( source[n] == '\n' )
            break;
    }

    tokenType   = ttOnelineComment;
    tokenLength = n+1;

    return true;
}

Of the last part of "source" is a comment such that the for loop breaks when n == sourceLength (instead of finding a line feed character), tokenLength gets assigned a value one greater than sourceLength.

It seems incorrect that the returned "tokenLength" should exceed the "sourceLength".

Of course you're right.

But how on earth did you find this bug? :)

I had written some preprocessing/intellisense code. One of my functions "SkipCommentsAndWhitespace" sometimes would crash when the source ended in a single-line comment. The pointer to the source was increased passed the end of the source (as a result of adding a length returned from the asIScriptEngine::ParseToken function). I'll admit, it took me a little while to track it down

Fixed in revision 1890