I am using DevIL to load an image, and then manipulating the image data to set the alpha value of all pixels under (4, 4, 4) to 0, or manual color keying. Afterwards, I enable alpha testing and draw the image. However, the transparent areas do not show up transparent, and are still drawn. Here is the color keying code:

	if ( transparent == true )
	{
		TextureData = new ILubyte[Width * Height * 4];
		ilCopyPixels( 0, 0, 0, Width, Height, 1, IL_RGBA, IL_UNSIGNED_BYTE, TextureData );

		TexturePixels = (DWORD *)TextureData;

		index = 0;
		while ( index < Width * Height )
		{
			// Loop through and test all pixels
			BYTE Red = TexturePixels[index] >> 24, Green = ( TexturePixels[index] << 8 ) >> 24, Blue = ( TexturePixels[index] << 16 ) >> 24;
			if ( Red <= 4 && Green <= 4 &&
				Blue <= 4 )
			{
				NewPixel = 0;
				NewPixel = ( Red << 24 ) || ( Green << 16 ) || ( Blue << 8 ) || 0;
				TexturePixels[index] = NewPixel;
			}

			index++;
		}

		ilSetPixels( 0, 0, 0, Width, Height, 1, IL_RGBA, IL_UNSIGNED_BYTE, TextureData );

		delete TextureData;
	}

Can anyone figure out what's wrong here?

Quote:

BYTE Red = TexturePixels[index] >> 24, Green = ( TexturePixels[index] << 8 ) >> 24, Blue = ( TexturePixels[index] << 16 ) >> 24;

I can't say for sure, but that might cause undefined behaviour. Try this instead:

BYTE Red = TexturePixels[index] >> 24, Green = ( TexturePixels[index] >> 16 ) & 0xFF, Blue = ( TexturePixels[index] >> 8 ) & 0xFF;

This does the same thing while avoiding shifting the same variable twice in one line.

The shift line should be fine. The shift operator produces a new value, it does not modify the existing variable, so double shifting should be fine (although I don't have a copy of the standard to confirm this). Although from a stylistic point of view I would split it into three lines to aid readability.

I can't see anything wrong with the code you posted. Are you sure you have your endianness correct (i.e. is red stored in the 8 MSBs of the dword)?

Have you actually set the alpha test function? The default is GL_ALWAYS, so just enabling alpha testing won't actually have any effect, except possibly to slow your program down! Make sure you call glAlphaFunc(GL_GREATER, 0).

A couple of other stylistic comments/errors:
You have a redundant line: NewPixel = 0;. The compiler ought to be warning you about this.
I'd even go so far as to say that NewPixel is redundant. Just make the line: TexturePixels[index] = (Red << 24) || (Green << 16) || (Blue << 8) || 0.
You allocate TextureData with operator new[] but delete it with operator delete. You must match operator new with operator delete and operator new[] with operator delete[]. In this case you should change 'delete TextureData;' to 'delete[] Texture Data'.

Enigma

Ah, thanks for the few stylistic things that I missed :).

Anyways, I rewrote the code to get the RGB values from different bytes, based on a message box I was popping up to get the color of the upper-left DWORD. The message box had 4 values for the 4 bytes:

255 0 0 0

Which leads me to believe that 255 is alpha, and the others are R, G, and B. After changing the code to accomendate that, nothing changed. All the colors stayed the same, alpha stayed at 1.0.

I believe that the alpha test function is correct:

glEnable( GL_ALPHA_TEST );
glAlphaFunc( GL_GREATER, 0.1 );

This worked when I loaded a TGA file with predefined alpha values.