Follow up the previous post. This time I tried to create a material graph editor in my mini Vulkan engine. (Almost no “Vulkan” point in this post though)

I used to just expose a few “fixed” parameters like diffuse color, bump scale, opacity...etc in an editor. That would be much easier but I want to practice more.

So I gave myself a challenge to build a material graph system from scratch. Since I’m not an expert on GUI, it looks like something from 90s lol. But at least it processes my logic well as shown in the test video.

*Disclaimer: This is just a personal practice of engine programming. Seeking something else if you're for game developing.

Git link: https://github.com/EasyJellySniper/Unheard-Engine​

Material Graph System

It is a trend in modern game engines. In the past, the shader calculations are usually “fixed” in the shader file. And programmers expose a few attributes for adjustment, such as diffuse color, specular color, emissive intensity...etc. Despite this makes implementation easier, it somehow gives limitation to non-programmer users.

So we have material graph system today! Unreal engine is a good example of this system. The way they utilizing their material input nodes are still fixed in their rendering pipeline. But before these happen, we can customize our logic in the graph. Do things like blending different colors without a touch of shader template code. On the other words, a graph system provides some flexibilities within a finite framework.

Since I have not implemented something like “content browser” or “resource manager” editor, all materials are loaded in a single material dialog for now.

Fundamental of material graph

I believe there are libraries (either open-sourced or not) for this. But for practice purpose, I build it from scratch.

To me, a material graph should at least consist of:

• Node structure – Data for both runtime and GUI display.
• Shader template – The shader template that lets material translate into.
• HLSL translator – A workflow to convert graph state to HLSL code. (Or other shader languages your engine preferred)

So it’s some sort of auto-generating shader code based on the graph inputs.

Node structure

I have two classes. One for graph node, and another one for the pins. They’re the base classes for all nodes.

 // Graph Node Class in UHE, a node can contain multiple inputs and outputs, the name, this is a base class
class UHGraphNode : public UHObject
{
public:
    UHGraphNode(bool bInCanBeDeleted);
    ~UHGraphNode();

    // node functions
    virtual bool CanEvalHLSL() { return true; }
    virtual std::string EvalDefinition() { return ""; }
    virtual std::string EvalHLSL() = 0;
    virtual bool IsEqual(const UHGraphNode* InNode) { return GetId() == InNode->GetId(); }

    // data I/O override
    virtual void InputData(std::ifstream& FileIn) = 0;
    virtual void OutputData(std::ofstream& FileOut) = 0;

    // GUI function for lookup
#if WITH_DEBUG
    void SetGUI(UHGraphNodeGUI* InGUI);
    UHGraphNodeGUI* GetGUI() const;
#endif

    std::string GetName() const;
    UHGraphNodeType GetType() const;
    std::vector<std::unique_ptr<UHGraphPin>>& GetInputs();
    std::vector<std::unique_ptr<UHGraphPin>>& GetOutputs();
    bool CanBeDeleted() const;

protected:
    std::string Name;
    UHGraphNodeType NodeType;
    std::vector<std::unique_ptr<UHGraphPin>> Inputs;
    std::vector<std::unique_ptr<UHGraphPin>> Outputs;
    bool bCanBeDeleted;

#if WITH_DEBUG
    UHGraphNodeGUI* GUICache;
#endif
};

// Graph Pin Class in UHE, nullptr of node means it's not connected
class UHGraphPin : public UHObject
{
public:
    UHGraphPin(std::string InName, UHGraphNode* InNode, UHGraphPinType InType, bool bInNeedInputField = false);
    ~UHGraphPin();

    void SetPinGUI(HWND InGUI);
    bool ConnectFrom(UHGraphPin* InSrcPin);
    void ConnectTo(UHGraphPin* InDestPin);
    void Disconnect(uint32_t DestPinID = UINT32_MAX);

    std::string GetName() const;
    UHGraphNode* GetOriginNode() const;
    UHGraphPinType GetType() const;
    UHGraphPin* GetSrcPin() const;
    std::vector<UHGraphPin*>& GetDestPins();
    HWND GetPinGUI() const;
    bool NeedInputField() const;

private:
    std::string Name;

    // source pin connected from
    UHGraphPin* SrcPin;

    // dest pin connected to, used for deletion
    std::vector<UHGraphPin*> DestPins;

    // origin node that hold this pin
    UHGraphNode* OriginNode;

    // pin type
    UHGraphPinType Type;

    // pin GUI
    HWND PinGUI;

    // if need input text field
    bool bNeedInputField;
};

As shown in the code pieces, my node class basically holds the name, type, input and output pins.

And the pin class holds the name, type, node it belongs to, source pin and destination pins.

An input pin will have only one input, but an output pin can have multiple connections to the others..

Theoretically this structure can provide dynamic pin adding/removing feature. But let me revisit this in the future. For now, pins are need to be defined during initialization.

The function ConnectFrom(), ConnectTo(), and Disconnect() are responible for pin connections. Just some pointers setup so I won’t detail it. If pin types are mismatched, the connection will fail

One exception is my math node, its pin types are set as “any node”. Since per-component math operations is allowed in the shader. For example, float3(1,1,1) * 10 is possible. Validity for math node will be determined in compile time instead.

And one example of node structure initialization from my material node:

UHMaterialNode::UHMaterialNode()
	: UHGraphNode(false)
{
	Name = "Material Inputs";

	// declare the input pins for material node
	Inputs.resize(Experimental::UHMaterialInputs::MaterialMax);
	Inputs[Experimental::UHMaterialInputs::Diffuse] = std::make_unique<UHGraphPin>("Diffuse (RGB)", this, Float3Node);
	Inputs[Experimental::UHMaterialInputs::Occlusion] = std::make_unique<UHGraphPin>("Occlusion (R)", this, FloatNode);
	Inputs[Experimental::UHMaterialInputs::Specular] = std::make_unique<UHGraphPin>("Specular (RGB)", this, Float3Node);
	Inputs[Experimental::UHMaterialInputs::Normal] = std::make_unique<UHGraphPin>("Normal (RGB)", this, Float3Node);
	Inputs[Experimental::UHMaterialInputs::Opacity] = std::make_unique<UHGraphPin>("Opacity (R)", this, FloatNode);
	Inputs[Experimental::UHMaterialInputs::Metallic] = std::make_unique<UHGraphPin>("Metallic (R)", this, FloatNode);
	Inputs[Experimental::UHMaterialInputs::Roughness] = std::make_unique<UHGraphPin>("Roughness (R)", this, FloatNode);
	Inputs[Experimental::UHMaterialInputs::FresnelFactor] = std::make_unique<UHGraphPin>("Fresnel Factor (R)", this, FloatNode);
	Inputs[Experimental::UHMaterialInputs::ReflectionFactor] = std::make_unique<UHGraphPin>("Reflection Factor (R)", this, FloatNode);
	Inputs[Experimental::UHMaterialInputs::Emissive] = std::make_unique<UHGraphPin>("Emissive (RGB)", this, Float3Node);
}

All node classes are put under Runtime/Classes/GraphNode if you want more examples.

Node GUI

As you can see in the test video, a node consists of a few win32 controls. That’s why it may flicker, which is caused by InvalidateRect() as always. The connection lines are drawn on another bitmap, then blit back to window, so they won’t flicker.

But for win32 controls, I can’t just drag them without invalidate the rect. Otherwise it will be ghosting...I believe the perfect solution will be image-based node implementation, drawing all nodes in another bitmap, and do not rely on these controls.

(I'll appreciate if you can share the ideas to terminate the flickering! The WM_ERASEBKGND trick doesn't seem to solve my use case.)

Anyway I’m not detailing GUI here, as I might seek better GUI solutions. At least my node and node GUI class are separated. So I don't mind a GUI reconstruction in the future!

Shader Template

For now, I've only applied the graph system to the base pass pixel shader.

//%UHS_TEXTUREDEFINE

UHMaterialInputs GetMaterialInput(float2 UV0)
{
	// material input code will be generated in C++ side
	//%UHS_INPUT
}

void BasePS(VertexOutput Vin
	, bool bIsFrontFace : SV_IsFrontFace
	, out float4 OutColor : SV_Target0
	, out float4 OutNormal : SV_Target1
	, out float4 OutMaterial : SV_Target2
	, out float4 OutEmissive : SV_Target3
	, out float OutMipRate : SV_Target4)
{
	// ... referring BasePixelShader.hlsl for details
}

System will load this template file, and alternate the %UHS_TEXTUREDEFINE and %UHS_INPUT identifier in memory.

• %UHS_TEXTUREDEFINE – Defines the texture and sampler registers.
• %UHS_INPUT – Graph calculations are mainly here.

An example of translation:

Texture2D Node_1324 : register(t9);
Texture2D Node_1340 : register(t10);
Texture2D Node_1347 : register(t11);
SamplerState DefaultAniso16 : register(s12);

UHMaterialInputs GetMaterialInput(float2 UV0)
{
	// material input code will be generated in C++ side
	float Node_161796 = 0.50f;
	float Node_130026 = 1.00f;
	float Node_135323 = 0.00f;
	float4 Result_1347 = Node_1347.Sample(DefaultAniso16, UV0) * 2.0f - 1.0f;
	float4 Result_1340 = Node_1340.Sample(DefaultAniso16, UV0);
	float3 Node_1330 = float3(0.50f, 0.50f, 0.50f);
	float Node_1327 = 1.00f;
	float4 Result_1324 = Node_1324.Sample(DefaultAniso16, UV0);
	float3 Node_1314 = float3(1.00f, 1.00f, 1.00f);

	UHMaterialInputs Input = (UHMaterialInputs)0;
	Input.Diffuse = (Node_1314 * Result_1324.rgb).rgb;
	Input.Occlusion = Node_1327.r;
	Input.Specular = (Node_1330 * Result_1340.rgb).rgb;
	Input.Normal = (Node_1327 * Result_1347.rgb).rgb;
	Input.Opacity = 1.0f;
	Input.Metallic = 0.0f;
	Input.Roughness = Node_135323.r;
	Input.FresnelFactor = Node_130026.r;
	Input.ReflectionFactor = Node_161796.r;
	Input.Emissive = float3(0,0,0);
	return Input;
}

After some string replacements, system will output this temporary file. So that DXC can compile and convert them to SPIR-V shader.

HLSL Translator

Most excited part for me! In my graph node class, there are CanEvalHLSL(), EvalDefinition(), EvalHLSL(). They will be used for translating. As like how they’re named:

• CanEvalHLSL() – Check if this node can be translated, default as true. Must be overriden for customizing rules.
• EvalDefinition() – To evaluate local definitions. This can be useful for preventing duplications. For example, if a texture node is outputting to multiple sources, it should still be only sampled once.
• EvalHLSL() – Mainly for returning calculations.

To translate a graph, I can either traverse from left-to-right (top-down) or right-to-left (bottom-up).

I go with bottom-up method of course. Imagining you did a lot of node connections in the graph, and forgot to connect it to material inputs? It would be a waste to translate them by top-down method in such case.

Also, bottom-up implementation can be easily done with some sort of recursion. Since I only designed 4 types of graph nodes at the moment, I’ll roughly go through all of them.

Material Node:

std::string UHMaterialNode::EvalHLSL()
{
	if (!CanEvalHLSL())
	{
		return "";
	}

	// parse definition
	std::vector<std::string> Definitions;
	std::unordered_map<uint32_t, bool> DefinitionTable;

	for (int32_t Idx = 0; Idx < Experimental::UHMaterialInputs::MaterialMax; Idx++)
	{
		CollectDefinitions(Inputs[Idx].get(), Definitions, DefinitionTable);
	}

	// Calculate property based on graph nodes
	std::string EndOfLine = ";\n";
	std::string TabIdentifier = "\t";
	std::string Code;

	for (int32_t Idx = static_cast<int32_t>(Definitions.size() - 1); Idx >= 0; Idx--)
	{
		if (Idx == Definitions.size() - 1)
		{
			Code += Definitions[Idx];
		}
		else
		{
			Code += TabIdentifier + Definitions[Idx];
		}
	}

	Code += "\n\tUHMaterialInputs Input = (UHMaterialInputs)0;\n";

	// Diffuse
	if (UHGraphPin* DiffuseSrc = Inputs[Experimental::UHMaterialInputs::Diffuse]->GetSrcPin())
	{
		Code += "\tInput.Diffuse = " + DiffuseSrc->GetOriginNode()->EvalHLSL() + ".rgb" + EndOfLine;
	}
	else
	{
		Code += "\tInput.Diffuse = 0.8f" + EndOfLine;
	}
	
	// .... other node translation .... //
}

The texture/sampler register will be added in UHMaterial::GetTextureDefineCode(), I'll skip them here. Anyway, the translation of a material node starts from my UHMaterialNode::EvalHLSL().

• It will collect the local parameters recursively, and utilize a table for lookup. So there won't be a redefinition.
• Then, translate each input nodes by calling source pin's EvalHLSL(). The example above is the diffuse input. If it's not connected, it will use a default value.

What if stack overflow happens during recursion? I have not tested what's the depth that can break the system yet. Also human edits are unlikely to reach the limit.

But if stack overflow really happened, I will just implement this with other structures.

Parameter Node:

// UH parameter node structure
template <typename T>
class UHParameterNode : public UHGraphNode
{
public:
	UHParameterNode(T InitValue)
		: UHGraphNode(true)
		, DefaultValue(InitValue)
	{

	}

	void SetValue(T InValue)
	{
		DefaultValue = InValue;
	}

	T GetValue() const
	{
		return DefaultValue;
	}

	virtual std::string EvalHLSL() override
	{
		// return the node name directly, value should be defined in EvalDefinition()
		if (CanEvalHLSL())
		{
			// eval from an exist input
			if (Inputs[0]->GetSrcPin() != nullptr && Inputs[0]->GetSrcPin()->GetOriginNode() != nullptr)
			{
				return Inputs[0]->GetSrcPin()->GetOriginNode()->EvalHLSL();
			}

			return "Node_" + std::to_string(GetId());
		}

		return "";
	}

	virtual void InputData(std::ifstream& FileIn) override
	{
		FileIn.read(reinterpret_cast<char*>(&DefaultValue), sizeof(DefaultValue));
	}

	virtual void OutputData(std::ofstream& FileOut) override
	{
		FileOut.write(reinterpret_cast<const char*>(&DefaultValue), sizeof(DefaultValue));
	}

protected:
	T DefaultValue;
};

UHFloat3Node::UHFloat3Node(XMFLOAT3 Default)
	: UHParameterNode<XMFLOAT3>(Default)
{
	Name = "Float3";
	NodeType = Float3;

	Inputs.resize(4);
	Inputs[0] = std::make_unique<UHGraphPin>("Input", this, Float3Node);
	Inputs[1] = std::make_unique<UHGraphPin>("X", this, FloatNode, true);
	Inputs[2] = std::make_unique<UHGraphPin>("Y", this, FloatNode, true);
	Inputs[3] = std::make_unique<UHGraphPin>("Z", this, FloatNode, true);

	Outputs.resize(1);
	Outputs[0] = std::make_unique<UHGraphPin>("Result", this, Float3Node);
}

std::string UHFloat3Node::EvalDefinition()
{
	// from input, skip definition
	if (Inputs[0]->GetSrcPin() && Inputs[0]->GetSrcPin()->GetOriginNode() != nullptr)
	{
		return "";
	}

	// consider input node
	const std::string InputX = (Inputs[1]->GetSrcPin() && Inputs[1]->GetSrcPin()->GetOriginNode()) ? Inputs[1]->GetSrcPin()->GetOriginNode()->EvalHLSL() : UHUtilities::FloatToString(DefaultValue.x);
	const std::string InputY = (Inputs[2]->GetSrcPin() && Inputs[2]->GetSrcPin()->GetOriginNode()) ? Inputs[2]->GetSrcPin()->GetOriginNode()->EvalHLSL() : UHUtilities::FloatToString(DefaultValue.y);
	const std::string InputZ = (Inputs[3]->GetSrcPin() && Inputs[3]->GetSrcPin()->GetOriginNode()) ? Inputs[3]->GetSrcPin()->GetOriginNode()->EvalHLSL() : UHUtilities::FloatToString(DefaultValue.z);

	// E.g. float3 Node_1234 = float3(0.0f, 0.0f, 0.0f);
	return "float3 Node_" + std::to_string(GetId()) + " = float3(" + InputX + "f, " + InputY + "f, " + InputZ + "f);\n";
}

For parameter nodes, they all have a base UHParameterNode template class. The rule is simple at the moment, return default value or evaluating from the input.

The EvalHLSL() of a parameter node simply returns its variable name, which is defined in EvalDefinition(). So if there are multiple destinations using this node, there is still only one instance of this node.

Texture Node:

bool UHTexture2DNode::CanEvalHLSL()
{
	if (SelectedTextureName.empty())
	{
		return false;
	}

	return true;
}

std::string UHTexture2DNode::EvalDefinition()
{
	// Eval local definition for texture sample, so it will only be sampled once only
	// float4 Result_1234 = Node_1234.Sample(Sampler, UV);
	// don't worry about texture definition, that's handled by material class
	if (CanEvalHLSL())
	{
		const std::string IDString = std::to_string(GetId());
		std::string UVString = GDefaultTextureChannel0Name;
		if (Inputs[0]->GetSrcPin() && Inputs[0]->GetSrcPin()->GetOriginNode())
		{
			UVString = Inputs[0]->GetSrcPin()->GetOriginNode()->EvalHLSL();
		}

		// consider if this is a bump texture and insert decode
		// @TODO: Remove this temporary method
		const bool bIsBumpTexture = UHUtilities::FindByElement(BumpTextures, SelectedTextureName);
		const std::string BumpDecode = bIsBumpTexture ? " * 2.0f - 1.0f" : "";

		return "float4 Result_" + IDString + " = Node_" + IDString + ".Sample(" + GDefaultSamplerName + ", " + UVString + ")" + BumpDecode + ";\n";
	}

	return "[ERROR] Texture not set.";
}

std::string UHTexture2DNode::EvalHLSL()
{
	if (CanEvalHLSL())
	{
		// @TODO: Consider channel variants
		const std::string IDString = std::to_string(GetId());
		return "Result_" + IDString + ".rgb";
	}

	return "[ERROR] Texture not set.";
}

Texture node will have no local definition, as texture needs to be defined in global scope. I've not designed “system input” node in the graph yet. So it's always using UV0 for now.

The texture sampling will be done in EvalDefinition(). And it will consider the bump texture input. Although the current method is temporary, it will do a bump decode when necessary.

Math Node:

bool UHMathNode::CanEvalHLSL()
{
	if (Inputs[0]->GetSrcPin() == nullptr || Inputs[1]->GetSrcPin() == nullptr)
	{
		return false;
	}

	return GetOutputPinType() != VoidNode;
}

std::string UHMathNode::EvalHLSL()
{
	if (CanEvalHLSL())
	{
		std::string Operators[] = { " + "," - "," * "," / " };
		return "(" +  Inputs[0]->GetSrcPin()->GetOriginNode()->EvalHLSL() + Operators[CurrentOperator] + Inputs[1]->GetSrcPin()->GetOriginNode()->EvalHLSL() + ")";
	}

	return "[ERROR] Type mismatch or input not connected.";
}

Looks straightforward. It's just returing [left operand] (operator) [right operand]. But will check if the pin types are matched. (With an exception of float type for per-component math operations)

By testing, current method seems solid enough.

In the future, I’ll just extend more nodes and functionalities based on the needs. As a very first version of my node system, they're still lack of some conveniences. Such as multiple output pins.

Also there must be bugs that I fail to test them myself.

Compilation Behaviour

My goal is to not compiling shaders at all in release mode, so the compiling will only be executed in debug mode. So the timing to rebuild shader is important. Now UHE will recompile shader with the following circumstances:

• When user clicks compile button in the material editor.
• When non-material shaders are modified.
• When shader include files are modified.
• When shader template files are modified.

Sometimes it doesn’t even need a recompiling despite the graph is modified.

For example, selecting another texture with an exist node. The descriptor should remain the same in this case. It just needs to rebind the texture parameter without a compilation.

Since I have not implemented material caches, it won't trigger a recompiling if there is a change in file system for materials. I shall solve this in the future too.

Generation of the default graph & Saving the material graph

UHMaterial::GenerateDefaultMaterialNodes() creates the default graph after the material is imported. Just some node initialization and connection based on the inputs from FBX material.

For saving the graph data, it's formatted as follow

• Number of material inputs.
• Connection state, the index to the node container array.
• Number of all edit nodes, their types and their values. Which is why there are InputData/OutputData function overrides in node classes.
• Number of an edit node's inputs, and its connection state.

When loading the graph data, it will spawn the nodes based on the type and sync the values. Then, restore the connection states for all nodes.

And I only save the connection state for inputs, no need to save the states for outputs. One-direction reconstructions will suffice.

Future Work & Summary

Huge change this time! Because I’ve never experienced writing a graph system from scratch. But it was still very funny to me :D

More node functions, better GUIs, apply material shaders to more renderings, editor changes, misc optimizations…all these can be my future works.

Cheers!