I'm starting out learning graphics programming with DX11 and Win32 and so far I haven't found many quality resources. At the moment I'm trying to string together a few tutorrials (some based on DX10) to try and learn the basics. The tutorial I was reading used a D3DX function to compile shaders but as far as I understand, this library is deprecated? After some digging, I found the D3DCompileFromFile() function which I'm trying to use.

Here is my c++ code:

HINSTANCE d3d_compiler_lib = LoadLibrary("D3DCompiler_47.DLL");
assert(d3d_compiler_lib != NULL);

typedef HRESULT (WINAPI *d3d_shader_compile_func)(
    LPCWSTR,
    const D3D_SHADER_MACRO *,
    ID3DInclude *,
    LPCSTR,
    LPCSTR,
    UINT,
    UINT,
    ID3DBlob **,
    ID3DBlob **);

d3d_shader_compile_func D3DCompileFromFile = (d3d_shader_compile_func)GetProcAddress(
    d3d_compiler_lib,
    "D3DCompileFromFile");
assert(D3DCompileFromFile != NULL);

ID3D10Blob *vs, *ps;

hresult = D3DCompileFromFile(
    L"basic.shader",
    NULL,
    NULL,
    "VertexShader",
    "vs_4_0",
    0,
    0,
    &vs,
    NULL);
assert(hresult == S_OK); // Fails here

hresult = D3DCompileFromFile(
    L"basic.shader",
    NULL,
    NULL,
    "PixelShader",
    "ps_4_0",
    0,
    0,
    &ps,
    NULL);
assert(hresult == S_OK);

FreeLibrary(d3d_compiler_lib);

In the failing assertion, hresult is 'E_FAIL', which according to MSDN, means: "Attempted to create a device with the debug layer enabled and the layer is not installed." I'm a bit lost at this point. I am also not 100% sure my D3DCompileFromFile signature is correct... It does match the version on MSDN though.

Any ideas as to why this might be failing? I tried putting in the wrong file name and got (as expected) hresult == D3D11_ERROR_FILE_NOT_FOUND. So at least this is some indication that I haven't totally screwed up the function call.

For reference, here is the shader file. I was able to compile it successfully using an online HLSL compiler.

struct VOut
{
    float4 position : SV_POSITION;
    float4 color : COLOR;
};

VOut VertexShader(float4 position : POSITION, float4 color : COLOR)
{
    VOut output;

    output.position = position;
    output.color = color;

    return output;
}


float4 PixelShader(float4 position : SV_POSITION, float4 color : COLOR) : SV_TARGET
{
    return color;
}

Thanks for your time

So that description of "E_FAIL" only applies when you're creating a D3D11 device using D3D11CreateDevice. For D3DCompileFromFile, that value most likely means that the compiler hit and error when attempting to compile the code that you gave it. To find out what that error is, you have to pass the address of an ID3DBlob* as the "ppErrorMsgs" argument (the last one). You can then extract the error messages from that blob. Here's a quick example of how to take that error message and output it to a message box:
 

ID3DBlob* compiledShader = nullptr;
ID3DBlob* errorMessages = nullptr;
HRESULT hr = D3DCompileFromFile(path, defines, nullptr, functionName,
                                profile, 0, 0, &compiledShader, &errorMessages);
if(FAILED(hr) && errorMessages)
{            
    const char* errorMsg = (const char*)errorMessages->GetBufferPointer();
    MessageBox(nullptr, errorMsg, L"Shader Compilation Error", MB_RETRYCANCEL);
}

Just make sure that you call Release() on the error message blob when you're done with it. Also, ppErrorMsgs might give you back some data even if the compile succeeds, which usually happens when you have compiler warnings but no errors.

One other thing that I wanted to point out: you don't need to use LoadLibrary and GetProcAddress to call the D3DCompiler functions if you don't want to. Instead you can link to the import lib (d3dcompiler.lib), and include d3dcompiler.h to get the function prototypes.

By the way: I think that your shaders are failing to compile because "VertexShader" and "PixelShader" are reserved keywords in HLSL. Try renaming your functions to something else.

One further comment is related to the specific error you mentioned, where the debug layer isn't installed.  You need to install the "Graphics Tools", which is an optional feature in Windows 10.  That will allow you to create a device with the debug layer enabled.

Also, as for tutorials, this one is pretty good: http://www.rastertek.com/tutdx11.html

On 07/03/2018 at 1:49 AM, MJP said:

So that description of "E_FAIL" only applies when you're creating a D3D11 device using D3D11CreateDevice. For D3DCompileFromFile, that value most likely means that the compiler hit and error when attempting to compile the code that you gave it. To find out what that error is, you have to pass the address of an ID3DBlob* as the "ppErrorMsgs" argument (the last one). You can then extract the error messages from that blob. Here's a quick example of how to take that error message and output it to a message box:
 

ID3DBlob* compiledShader = nullptr;
ID3DBlob* errorMessages = nullptr;
HRESULT hr = D3DCompileFromFile(path, defines, nullptr, functionName,
                                profile, 0, 0, &compiledShader, &errorMessages);
if(FAILED(hr) && errorMessages)
{            
    const char* errorMsg = (const char*)errorMessages->GetBufferPointer();
    MessageBox(nullptr, errorMsg, L"Shader Compilation Error", MB_RETRYCANCEL);
}

Just make sure that you call Release() on the error message blob when you're done with it. Also, ppErrorMsgs might give you back some data even if the compile succeeds, which usually happens when you have compiler warnings but no errors.

One other thing that I wanted to point out: you don't need to use LoadLibrary and GetProcAddress to call the D3DCompiler functions if you don't want to. Instead you can link to the import lib (d3dcompiler.lib), and include d3dcompiler.h to get the function prototypes.

By the way: I think that your shaders are failing to compile because "VertexShader" and "PixelShader" are reserved keywords in HLSL. Try renaming your functions to something else.

Very late reply, but thank you! You were right about the reserved words!