Basically I want a clean way to be able to use one shader program with multiple rendering APIs. As described in the linked topic above, my current solution to this problem is to match and modify specific tokens in my intermediate "language" in order to generate the final HLSL/GLSL code. This may sound good ?n paper, but as you can see (in the topic above) the shader code is bloated, ugly and highly unextendable (this is why I'm trying to solve this issue with a different approach).
After asking the public, I've decided to give the language writing a try (thanks to L. Spiro and a colleague of mine - G. Totev).
The idea is to write a language that could be translated to HLSL/GLSL/(something else). I was really scared of that task because... well, it is about writing a compiler and certainly I don't know (and still don't) how to write one.
Thankfully, there is a pair of tools - Flex and Bison (their ancestors are lex and yacc). These tools are used to describe the language.
Flex is used for finding (matching) keywords, strings, literals, identifiers, and Bison is used for matching those tokens in expressions (in other words describing the grammar of the language by using context free grammars). The Output of these tools is a set of C/C++ function(s) that knows how to parse your language. But more on C/C++ integration with Flex/Bison later (or maybe in another Journal).
After grinding a lot I found out that this isn't that hard of a task for my humble needs. The big problem was finding good resources on the topic. After hours of researching I found these :
http://people.seas.harvard.edu/~bwaggoner/writeups/jumpstart/flexbison/jumpstart_flexbison.pdf
http://epaperpress.com/lexandyacc/download/LexAndYaccTutorial.pdf
(a quick hint: add yacc to your search query in order to increase the chance for correct results).
I'm currently in the process of developing that language. This is still not a complete solution, but the hard work is almost done.
The language can :
- Deduce the type of an expression ( for example the languages knows that matrix * vector in HLSL should be mul(matrix, vector) )
- Variable shadowing.
- Is able to produce some HLSL/GLSL (the shading stage specifics aren't yet supported, but it's just a matter of time. Currently there are input/output varyings, global uniforms, vertex attributes).
- Error checking is done only if the code generation depends on it. I want the code simple, why I want that is described below.
The code can be found here : https://github.com/ongamex/playground/tree/master/lang KEEP IN MIND THAT IT IS A PLAYGROUND REPOSITORY, but you can look at it if you're curious.
So what is the final goal?
The ultimate goal is to provide the base functionality (not a complete solution), that is easily extendable by others.
There are similar projects that either dead or their code is too complicated which basically means that insane amount of time is needed to create a modification. Currently the project doesn't meet this requirement (but the code is still very simple). In order to achieve that I must find the best architecture for the code and write (or find) a good tutorial about Bison.
And finally a quick demo of the language :
// KEEP IN MIND THAT THIS IS NOT A REAL SHADER :)// GLSL like declarations with nice flavor of HLSL's semantics.attribute vec3f a_pos : POSITION;uniform Texture2D diffuseTex;out vec2f uv; //output varyingin mat4f world; // input varyingvoid main() { mat4f m; vec4f v = vec4f(5); int x = lerp(0.3, 0.0, 1.0); uv = 2; // the language still doesn't care about types, but... // HERE it does // the compiler knows that we are multiplying a matrix by a ?????. // And outputs the correct result m * lerp(0.3, 5.0, 10.0) * vec4f(0.0); m * mix(0.3, vec4f(0), vec4f(1.0)) // if-else branching works fine... if(x>0) v = -(world * -v.xxxy * +(getm() * v)); // member access and swizzles. else if(x<0) x = 10; vertex_output = v; }mat4f getm() { return; } // the language doesn't care what you're are actually returning, but you can get that error from the hlsl/glsl compiler.And so far the output HLSL(I will skip the GLSL) looks like this(with a bit of shuffling) :
uniform Texture2D diffuseTex;uniform sampler diffuseTex_sgeSS;float4x4 getm() { return;}struct SGE_SHADER_RESULT { float2 uv : uv; float3 vertex_output : SV_Position;};SGE_SHADER_RESULT main(float3 a_pos : POSITION, float4x4 world : world) { SGE_SHADER_RESULT sge_shader_result; float4x4 m; float4 v=float4(5); int x=lerp(0.30, 0.0, 1.0); sge_shader_result.uv = 2; if(x > 0)v = -(mul(world, -v.xxxy) * (mul(getm(), v))); else if(x < 0)x = 10; sge_shader_result.vertex_output = v; mul(m * lerp(0.30, 5.0, 10.0), float4(0.0)); mul(m, lerp(0.30, float4(0), float4(1.0))); return sge_shader_result;}I will post more updates about the language here or In a new journal. Feel free to ask me anything, report an error, and give any type of feedback!
L. Spiro