I have been trying to create a BlendState for my UI text sprites so that they are both alpha-blended (so you can see them) and invert the pixel they are rendered over (again, so you can see them).

In order to get alpha blending you would need:

SrcBlend = SRC_ALPHA
DestBlend = INV_SRC_ALPHA

and in order to have inverted colours you would need something like:

SrcBlend = INV_DEST_COLOR
DestBlend = INV_SRC_COLOR

and you can't have both.

So I have come to the conclusion that it's not possible; am I right?

That second results in:

RenderTarget = PixelShaderOut * (1-RenderTarget) + RenderTarget * (1-PixelShaderOut)

Which is something.. Quite different to simply an inverted color. 

You want the destination color to be inverted, then have your sprite alpha blended over the top? So, alpha=0 produces an inverted background, alpha=1 produces the sprite, and alpha = 0.5 is half way between the two? 

25 minutes ago, Hodgman said:

You want the destination color to be inverted, then have your sprite alpha blended over the top? So, alpha=0 produces an inverted background, alpha=1 produces the sprite, and alpha = 0.5 is half way between the two? 

Yes.  Basically I want to see the text regardless of colour of the pixel behind (this is only for debug text as I don't think that would look much good in production).

Ok that would be something like:

RenderTarget.rgb = PixelShaderOut.rgb * PixelShaderOut.a + (1 - RenderTarget) * (1-PixelShaderOut.a)

Which yeah, is too complex for the fixed function blending unit. You would have to do two passes (one to invert the background, then a second to blend the text over it), or do the blending entirely in a shader instead of the blending unit. 

... Actually, using some algebra you can use the fact that:

(1-x)*(1-y) == x(y-1)+1-y

To rearrange this blend equation to:

RenderTarget.rgb = PixelShaderOut.rgb * PixelShaderOut.a + RenderTarget.rgb * (PixelShaderOut.a-1) + 1 -PixelShaderOut.a

...which is still too complex for the blend unit, but if you edit the end of your shader from:

return result;

To this, which moves some of that equation to the shader:

return float4(result.rgb * result.a + 1 - result.a, result.a - 1)

Then you can simplify the blend state to:

RenderTarget.rgb = PixelShaderOut.rgb + RenderTarget.rgb * PixelShaderOut.a

Which is just: SrcFactor = one, DstFactor = SrcAlpha! 

Whoa!  That's amazing.  I'll try this tonight and get back to you.

Many thanks!

OK, so that's not working.  This is what it tooks like (text is bottom left):

Here is the blend state creation code:

D3D11_BLEND_DESC blendDesc = {};
blendDesc.RenderTarget[0].BlendEnable = TRUE;
blendDesc.RenderTarget[0].RenderTargetWriteMask = D3D11_COLOR_WRITE_ENABLE_ALL;
blendDesc.RenderTarget[0].BlendOp = D3D11_BLEND_OP_ADD;
blendDesc.RenderTarget[0].SrcBlend = D3D11_BLEND_ONE;
blendDesc.RenderTarget[0].DestBlend = D3D11_BLEND_SRC_ALPHA;
blendDesc.RenderTarget[0].BlendOpAlpha = D3D11_BLEND_OP_ADD;
blendDesc.RenderTarget[0].SrcBlendAlpha = D3D11_BLEND_ONE;
blendDesc.RenderTarget[0].DestBlendAlpha = D3D11_BLEND_ZERO;
ThrowIfFailed(
    d3dDevice->CreateBlendState(&blendDesc, s_invertedBlendState.GetAddressOf()),
    "create Text inverted blend state"
);

and here is the pixel shader:

Texture2D txDiffuse : register(t0);
SamplerState samp : register(s0);

float4 main(PS_IN_PosColTex input) : SV_TARGET
{
    float4 color = txDiffuse.Sample(samp, input.tex);
    return float4(color.rgb * color.a + 1 - color.a, color.a - 1);
}

However on the screen the text has black boxes instead of white from the screenshot.

I use a stack of Scene objects with the UI Scene being on top.  In the render loop I clear a render target to transparent and each Scene renders their content onto the render target and finally the render target is written to the window render target (this is for future expand where post-processing is expected).  Would have some bearing on any of this?

If it's just for debugging purposes, why not just render the text in white with black background or something...?

13 minutes ago, vinterberg said:

If it's just for debugging purposes, why not just render the text in white with black background or something...?

 

Yep, that was my next step, however I thought I'd just ask to see if it was possible.