So basically i want to draw tree leaves into the shadowmap, to do that i would need to disable depth writes to zbuff and read depth value of current fragment to see if leaf (as its a texture with alpha) is closer to light than actual fragment.

Now i know that i cant write and read to the same fbo texture so i could sample depth (please note i write depth value to rgba tex), now when i try to draw an alpha tex i need to sample this alpha texture to see if it has alpha 1.0 to perform a check whenever this fragment is closer to the light, so i could update this rgba depth tex.(note that i have to disable depth test, so i actually can sample whole leaf polygon)

No i dont know how i could handle that. (reading and writing to same tex)

Hey WeirdCat, how are you?

I assume your post means something more to whoever follows you on the discord channel or anything but since a few days have gone by, I'll drop you a few lines.

_WeirdCat_ said:
i want to draw tree leaves into the shadowmap, … is closer to light than actual fragment.

Perhaps you are doing some deferred shading mumbo-jumbo but in the day I used to do those things I first drawn the shadowmap.

As such, there's no such thing as “another fragment” to compare to. The comparison is in a later step when the shadowmap is fully ready.

When drawing the shadowmap, leave zwrites on! It won't do much otherwise. You probably meant disable the color writes. In their basic formulations, shadowmaps are no different from regular rendering with no color.

_WeirdCat_ said:
please note i write depth value to rgba tex

I take for granted you do this for some compatibility reason or anything - perhaps to avoid a z lock?

_WeirdCat_ said:
note that i have to disable depth test, so i actually can sample whole leaf polygon

Uhm.. so you are writing your shadowmap and you want to self-shadow your leaves.

Just don't. The contribution of a shadowed leaf to the shadowmap is NOP. From the point of view of the light/shadow projector only the topmost/nearest pixel/fragment contributes to result. The Z compare op will sort your nearest fragment for you (it becomes quite more complicated for color).

I assume your post means something more to whoever follows you on the discord channel or anything

I dont follow gdnet abilities so i have no clue what you've just said.

If i enable depth testing it will occlude leaves behind leaves ;p

I just store depth i rgba byte texture cause i really have no option for ogl es, there is a halffloat tex extension, but it doesnt have to work on all devices, having 32 bit precision here i write depth to each tex pixel….

Anyway since theres no option to somehow write this into shadowmap in one pass(read framebuffer texture and compare it to calculated new depth), i use separated alpha shadowmap where i do this magic, then compare both shadowmaps using only the closest value from both maps.

Since its omnidir pointlight i use 13 textures per face ? thats why i wanted to avoid using separated framebuffer only for drawing alpha-shadow.

Hey WeirdCat, how are you?

Could have been worse :0

Actually i havent tried the results, cause now im dealing with adding collision , as soon as i write that ill check if this thing of mine works…..

_WeirdCat_ said:
If i enable depth testing it will occlude leaves behind leaves ;p

And… isn't that what you want when you draw a shadow map?

I may have been wrong here but what when you first draw the leaves above leaves below then you draw leves below they wont be considered in the final shadowmap thats why you have to sort blended primitivea from furthest to nearest right? I wont be sorting them during depth write stage…

Maybe for now ill see what sort of optimizations i will apply but for now i dont use any

Your ordering is a mind construct of the choices you have taken in advance. I don't support those.

Rear-to-front rendering is required only for correct blending of transparent surfaces. We are talking in the sense of `glEnable(GL_BLEND)`, the blendfunc and the blend terms (GL_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA), those things.

Your leaves are traslucent ie alpha tested (old GL_ALPHA_TEST and now shader discard). Correct rendering does not require ordering (as long as KISS). A lot of leaves go away with just transparency off and alpha test. It is sufficient for simple games.

As soon as blending enters the picture shadowmaps go awry! There are some “transparency multilayer shadowmaps” techniques around they are quite involved and when it comes to me their days are counted.

Yeah, i have the impression you just have not thought about using alpha testing instead blending? That's how it works the easy and fast way.
Blending depth does not work - only a layered / stochastic SM would. It's not about technical limitations.

But maybe you could use alpha testing for SM, and blending for camera view.

Blending depth does not work

Yet it woks…

I did the hard way having 13 fbos for each light (6 for normal depth write to shadowmap) another 6 to draw there alpha depth ) (why 6 its an omnidir light i draw scene 6 times) and one fbo to save depth comparsion, color etc.

Basically

draw to depth without alpha textures

draw only alphatextures to separate fbo

compare depths from both normal shadowmpa and alpha shadowmap finding closest one to light

etc.

here are some screenshots

please keep in mind i use 512x512 fbo size so its not quite sharp image

also dunno how to know when sdl image flips opened texture (seems PNG files are opened differently than others) there are also reddish semitransparent quads, i dont sort them for now they are marked as nodraw so they aint written into shadowmaps

_WeirdCat_ said:
Yet it woks…

hmm… let's make sure about the terms first:

Alpha blending: Mixing different colors, e.g. for transparency like your orange cubes. We can mix color, but if we do this for depth we only get an average distance, which falls between two objects so does not represent the distance to either. (My stochastic SM example was bad - i think multiple moments SM or convolution SM can make a density function so they can approximate transparency.)

Alpha testing: There is no transparency, but a texture decides if surface exists or not. And the result looks like from your screenshot, so i think you use this. But this would not require rendering two SMs, just discard fragments that should not exist, and there will be no write to depth buffer.

And usually trees and their shadow SMs are drawn with alpha testing. It works, because either there is a leaf or nothing.
But it does not support transparency to see ‘through’ the leaf a bit, or to filter its blocky edges. Thus the alpha texture should be high res to avoid blockyness, and for shadows filtering is used but flicker for animated trees is often still visible.

What you describe sounds more like using a projection method this: https://docs.unrealengine.com/en-US/Engine/Rendering/Materials/HowTo/ColoredTransluscentShadows/index.html​
But combining it with usual SM so it can handle occluders as well? I do not understand what you do with the extra fbo:

and one fbo to save depth comparsion, color etc.

However, the SMs store just one depth, so you can not determinate decreasing shadow as light goes through many translucent leafs?

I wonder if you really need to have this, or if faster alpha testing would be good enough.

ok just to be sure i could have done this better (as far as i remember gl_alpha_test is not present in ogles).

Anyway once again ill describe the thing i am doing, maybe i was using wrong words or didint explain everything.

SO look at thrid image theres a floor and two blocks (now imagine there is only one face for each block - the one that faces light), we set them as alpha textures. Imagine there is also a third block not visible on the img but its there..

in the first pass i write depth values to texture = (distance from light to vertex) / lightprojectionmat_zfar this is done with depth writes (and depth test set as gl_lequal) so only closest fragments are rendered and their distance to light is written to a fbo - here i draw only lit objects (without any transparent faces)

in second pass i draw only transparent objects (like this fence) to do that without sorting i disable depth writes and depth test - so every face is rendered, whenever texture texel is not transparent i write depth value (dst from light to vertex/ lightprjmatz_far) to texture. and here i found the bug… cause i compare depth from first pass to this depth, meaning whenever some lit object is further to light, the depth value is written, but when i have more than two transparent faces there can be a case when one transparet face that is further than another transparent face writes its depth above the one that should be closer, making further transparent face not lited by shadow from closer transparent face, so actually i need to use depth testing and depth mask writes to discard fragments that are fuly transparent… let me check the results mhm

yeah seems now its working like it should i dont even need to lookup for another shadow texture to compare alpha depths, however now im thinking if i really need second fbo.

seems like no for now, still have to use different shader for alpha textures but seems like i dont need to draw to another 6 fbos…. to test it, i need do some heavy recoding so ill be out for a little while ;]