Hi guys,

This is not strickly a games related question but I was hoping that some of you with more experience than me might be able to shed some light on why I am getting the following results.

Using Opengl, I was able to generate capture the depth of a scene around a viewer location and generate a flat rendering of this scene after using spherical coordinates to sample a previously generated cubemap containing the depth information of a scene.

The result was the resulting image,

At a first glance looks exactly what I was after. However, when this image (containing depths) is differentiated numerically in both the vertical and horizontal direction respectively, we get some artifacts,

You can see at the bottom how it appears to pick up the seams of the cubemap. These will be more or less obvious depending on the viewpoint location. I made sure that glEnable(GL_TEXTURE_CUBE_MAP_SEAMLESS) was included before sampling the cubemap.

My goal is to produce a rendering where each pixel contains the depth of a regular fixed azimuth and pitch. Sampling of the previously created cubemap is done using spherical coordinates (I am using the following fragment shader).

#version 440 core
uniform vec2 display_size;
uniform vec2 offset; // current viewer's camera yaw and pitch
uniform vec2 fov;
uniform samplerCube cube;
out float color;
void main()
{
   // Calculate angles used for sampling
   float u = (gl_FragCoord.x / display_size.x) * 2 - 1;
   float v = (gl_FragCoord.y / display_size.y) * 2 - 1;
   float theta = u * radians(fov.x/2) + radians(offset.x);    
   float phi   =  v * radians(fov.y/2) + radians(offset.y);
   // make sure that phi is between [-89.9, 89.9]
   phi = clamp(phi, radians(-89.9), radians(89.9));
   // calculate sampling coordinates
   vec3 dir;
   dir.x = -cos(phi) * cos(theta);
   dir.y = sin(phi);
   dir.z = cos(phi) * sin(theta);
   color = texture(cube, dir).r;
}  

Could this due to the sampling scheme I am using? (I do not think so as it appears to be sampling smoothly across the cubemap faces. Artifacts only appear when three faces are used, specifically the bottom Y- face). Is there anyway to resolve this?

Any comments, suggestions will be very welcome!!

Thanks,

M.

I believe the artifacts in your derivative images are inherent to the cube map representation. When you sample the cube map, the values are continuous (C0 continuity), but they are not continuous in their derivatives (not C1 continuous).

You also probably want to linearize the depth, since the depth buffer you sample will not contain the raw depth, but instead contains 1/depth. Any nonlinearities in the depth values could be causing lack of C1 continuity when calculating derivatives across the cube faces.

Many thanks @@Aressera !!

The depth is actually not the depth derived from the depth buffer but the one derived from finding the distance from the camera location to each pixel.

Hmmm,… what I find more concerning is what you mention about the discontinuities for the derivatives. Why wouldn't these appear when sampling across (i.e. when sampling the cubemap faces X+ X- Z+ Z-) the cubemap faces???

Also I have come across the following link about Dual-Paraboloid Reflections. Would this be able to solve this problem and if so, how would this be implemented???

Thanks again.

mllobera said:
Hmmm,… what I find more concerning is what you mention about the discontinuities for the derivatives. Why wouldn't these appear when sampling across (i.e. when sampling the cubemap faces X+ X- Z+ Z-) the cubemap faces???

The artifacts are most visible in the corners of the cube.
It helps to increase the resolution of the texture, so the corners cover less texels and the artifacts become smaller, but ther is no way to avoid these artifacts.
Also keep in mind that the bilinear texture filter itself is not C1.
Basically we have only low quality filtering with GPUs. If we need better quality, we need to implement a higher order filter ourselves. (Which is much slower than HW acceleration ofc.)

mllobera said:
Dual-Paraboloid Reflections.

The problem area moves from the cube vertices to the split diameter.

It might be better for your case eventually,
but i would just calculate the color from the spherical coordinates directly, not using any form of environment texture.
(You could still use a 1D gradient texture to achieve the desired coloring if needed)

Thanks @JoeJ

i would just calculate the color from the spherical coordinates directly, not using any form of environment texture.
(You could still use a 1D gradient texture to achieve the desired coloring if needed)

I was not able to understand your last suggestion. Could you elaborate a bit more?

mllobera said:
I was not able to understand your last suggestion. Could you elaborate a bit more?

Oh, never mind.
I was thinking you would render a terrain mesh, using the vertex normals for the cubemap to get the coloring.
But now after reading your code, i see the mountains are in the cubemap texture, and the spike artifacts at the bottom are probably cube edges.

Looking closeley, i think the problem is maybe in the texture. Make sure there is some color dilation across the UV boundary. It looks like you could fix it by painting green over the texture, at the right spots?
It does not look like the typical corner filtering artifact i was talking about. Your corners should be just green, so the artifact should not affect your example.

Otherwise i would guess something like texture wrapping mode is wrong, although i'm not sure if such mistake is possible using cube maps.

Pitching the view direction down might reveal a clue? Or cubemap width is not a power of two?

@JoeJ Forgive me for dropping the ball after your last reply…

Couple of things. The values that I have are actually depth values (well technically the actual distance from the camera to each pixel) so after taking the first derivative I am actually getting changes in these depth values (which I am interested in). These changes are meaningful hence having them appear because of a problem with the cubemap sampling is not a good thing for my purposes. I have tried selecting a nearest interpolation and changing the texture wrapping as you suggested (currently, CLAMP_TO_EDGE) but this has not improved the situation. You mentioned something about doing some “dilation across the UV boundary"??? How would that work?

Bottom line, if my cubemap stores distance from camera location, how should I sample it so that I can end up with a flat rendering that can be numerically differentiated (currently it works perfect except for the bottom edges as shown!).

mllobera said:
Bottom line, if my cubemap stores distance from camera location, how should I sample it so that I can end up with a flat rendering that can be numerically differentiated (currently it works perfect except for the bottom edges as shown!).

Sadly i have no experience yet with using cubemaps for omnidirectional shadowmaps. But while thinking about that, i assume it's eventually necessary to use distance to camera position instead distance to the camera projection plane as usual.
Using distance it should not matter that each cubemap face has a different projection plane.

But you do that already.

mllobera said:
You mentioned something about doing some “dilation across the UV boundary

I guess the dialtaion propasal is a dead end too. For cubemaps dilation should not be needed at all, but if so, artifacts would show on all edges of the cube.
However, your problem only shows at the bottum face.
You could use a closed scene, e.g. inside a room, so there is geometry also upwards. Just to verify that no artifacts appear on the top face either, even if it's not just sky.

Currently i would speculate you have a bug when setting up the 6 cameras for cubemap rendering. Maybe the bottom face projection has some offset, so the distance values do not align with the other faces.
Because only the bottom edges show the problem, it's maybe some bug which has nothing to do with filtering.

mllobera said:
so after taking the first derivative I am actually getting changes in these depth values (which I am interested in).

Looking at your visualization code again, i see you do not calculate derivatives there, so you store them in the cubemap instead, i guess.

If so, the derivatives at the edges of the cubemap images were calculated without knowledge of the distance from the adjacent faces.
If that's related to the problem, you could try some debugging ideas:
1. Turn the whole setup upside down, to see if the ertifacts now show at the top instead of the bottom.
2. Set the face camera projections to a fov a bit larger than 90 degrees, so adjacent distances ar given, but then crop and scale the images accordingly, so the cubemap has 90 degree fovs as expected.
3. Or store distance instead derivatives, but then calculate the derivatives from the sampled distances in the visualization shader.

You should also visualize the 6 renderings for the cubemap (or use a graphics debugger like RenderDoc), to check if the artifact is visible already there.
Then you would see if the artifact is only in the bottom image, or also in the 4 images adjacent to that.
This would be a good clue about the origin of the problem.

Many thanks @JoeJ for the follow up and the suggestions!!!

I will double check when generating the 6 cubemap faces… I have, in the past, save them independently and did not noticed anything BUT it is also true that I was not looking for that.

The derivatives are calculated AFTER I render parts of the cubemap (depending on the viewer viewpoint) flat, ie. after the first image I included in my message was generated. It is not done in GLSL but insted just running a ‘gradient’ filter on the resultant image that calculates differences horizontally and vertically (I am using Python Numpy ‘gradient()’ function is that make any sense to you).

I will double check some of your suggestions and report back with what I find. Many thanks again.