I'll keep this high level as I'm not the developer in question, but in our WebGL/3JS project we have some models which are straight tubes. We manipulate these to follow splines using shaders, so as far as the 'engine' is concerned they are straight tubes, then they get deformed at the rendering stage to follow their real path.

This means trying to employ picking to detect the mouse hovering over a model doesn't work - it picks the straight-tube version.

I gather a lot of geometrical stuff is done in GPU/shaders these days so I wondered if that means this is a common problem with some known solutions/ideas?

You can redraw each object (modified by shader) in different color (you store which object has which color on cpu,)

Then you read a pixel color from the screen, its faster than doong math (not to mention less complexity) and actually you can draw lest say 200 k objects and get thebresult pick in notime...

1 hour ago, KKTHXBYE said:

You can redraw each object (modified by shader) in different color (you store which object has which color on cpu,)

Then you read a pixel color from the screen, its faster than doong math (not to mention less complexity) and actually you can draw lest say 200 k objects and get thebresult pick in notime...

So to check I've understood this correctly, we render the entire scene an extra pass to a separate buffer, with color keyed to object-Id. Then we remove picking entirely [ O(1) complexity ] at the cost of an extra render pass.

I can see on hardware these days a render-pass is often trivially cheap. And you can presumably use some handy buffer format like X16... or if you wanted depth information too you could add it.

I dont know what x16 is but, remember that color precision and color component bit depth is cruical,

To use these 16,7 million objects, you need 8 bits per color component so it means: red, green, blue have to be with a size of 1 byte. I tell you that cause devices like mobile phones use 3bit color component representation as default, 

Now imagine 2 objects

Whenever you use separated buffer or a main buffer you clear it and leave it black (0,0,0) then yoh rendrr one ibject that has a color (0,0,1) and another that has color (0,0,2)

I dont know how dx represents colors but in opengl theyre floats so you pass a color of an object to shader like this:

SendToUniform("object_color", vec3(float(r_component)/255.0, float(g_component)/255.0,  float(b_component)/255.0);

As far as i remember youll read pixel data as unsigned chars, so actually you will compare to these 0..255 defined on cpu, 

Just set color depth precision to be 8 bits per component and picking will work

Can you 'pick' against that spline instead of the geometry? 

I do picking in the pixel shader that I use to draw my objects, without using an additional pass. My code is dx11 or dx12, but you might be able to employ a similar technique in your language.

I pass an "object number" down from the vertex shader to the pixel shader when i draw my objects.

I send the mouse coordinates to the pixelshader in a cbuffer (its in my cbeveryframe buffer that is updated each frame)

In the pixel shader, if the pixel coodinates are equal to the mouse coordinates, I construct a uint where the high 16 bits are the pixel depth and the low 16 bits are the data to send back, in this case data is the "object number"

I then write this number to a uav buffer using an interlockedmin function.

This causes the closest object at the mouse coordinates to have its "object number" to be written to the uav

Then in the cpu I read this uav buffer (well actually a cpu-map buffer that I copied the uav to), and take the low 16 bits of the uint to get the object number.

I send back various pieces of information about the object under the mouse back to the cpu using this techique by using different uint variables in that uav.buffer.

That pixel shader is also used to draw the object after the above piece of code.

Performance tip when sending things from the gpu to the cpu - dont try to read the same buffer in the cpu imediately after the cpu draw command that causes that buffer to be written by the gpu. Allow it a few frames by for example having 3 uav buffers (and their corresponding cpu-map buffers that they are copied to) that you rotate between.

This is because the draw in the gpu doesnt occur imediately it is issued in the cpu.

hit proxy rendering (i think thats what its called) works well, but there's also the stream output stage which you can pipe back out geometry that was created in the pipeline if your using directx, not sure how to do it for opengl though.

Ok, sorry, i just realized you said webgl. in that case i would go with the hit proxy rendering, i don't believe there's any other way in webgl, since all you have is the vertex and pixel shaders to work with. webgl2, based on opengl es 3.0, has the added geometry shader, but still don't think you'll be able to pipe geometry back out unless you render it to an image, which in that case you might as well just use the hit proxy rendering (render to depth buffer, then store/render object id to color buffer maybe, or if you can fit the pickable object id in the stencil buffer). if you could fit it in the stencil buffer, or render the id to a separate render target, you could do this in the same pass as your regular draw calls, but that would mean modifying all of your fragment shaders to do that, so it might be easier to just do it in a second pass

On 7/14/2018 at 5:57 PM, KKTHXBYE said:

I dont know what x16 is but, remember that color precision and color component bit depth is cruical,

To use these 16,7 million objects, you need 8 bits per color component so it means: red, green, blue have to be with a size of 1 byte. I tell you that cause devices like mobile phones use 3bit color component representation as default, 

Now imagine 2 objects

Whenever you use separated buffer or a main buffer you clear it and leave it black (0,0,0) then yoh rendrr one ibject that has a color (0,0,1) and another that has color (0,0,2)

I dont know how dx represents colors but in opengl theyre floats so you pass a color of an object to shader like this:

SendToUniform("object_color", vec3(float(r_component)/255.0, float(g_component)/255.0,  float(b_component)/255.0);

 

As far as i remember youll read pixel data as unsigned chars, so actually you will compare to these 0..255 defined on cpu, 

Just set color depth precision to be 8 bits per component and picking will work

By 'x16' I was referring to the idea of rendering to a non-RGB(A) format buffer. I thought both GL and DX supported a whole wealth of integer and float formats now with different numbers of channels so you can have a 16bit single-channel integer format?

On 7/15/2018 at 2:45 PM, Hodgman said:

Can you 'pick' against that spline instead of the geometry? 

I wondered about this and I'm sure it's feasible but we have other complications so it seems the render-based approach is preferable, or we'll be writing several such special-case logic.

On 7/15/2018 at 6:53 PM, CortexDragon said:

I do picking in the pixel shader that I use to draw my objects, without using an additional pass. My code is dx11 or dx12, but you might be able to employ a similar technique in your language.

 

I pass an "object number" down from the vertex shader to the pixel shader when i draw my objects.

I send the mouse coordinates to the pixelshader in a cbuffer (its in my cbeveryframe buffer that is updated each frame)

In the pixel shader, if the pixel coodinates are equal to the mouse coordinates, I construct a uint where the high 16 bits are the pixel depth and the low 16 bits are the data to send back, in this case data is the "object number"

I then write this number to a uav buffer using an interlockedmin function.

This causes the closest object at the mouse coordinates to have its "object number" to be written to the uav

Then in the cpu I read this uav buffer (well actually a cpu-map buffer that I copied the uav to), and take the low 16 bits of the uint to get the object number.

I send back various pieces of information about the object under the mouse back to the cpu using this techique by using different uint variables in that uav.buffer.

That pixel shader is also used to draw the object after the above piece of code.

 

Performance tip when sending things from the gpu to the cpu - dont try to read the same buffer in the cpu imediately after the cpu draw command that causes that buffer to be written by the gpu. Allow it a few frames by for example having 3 uav buffers (and their corresponding cpu-map buffers that they are copied to) that you rotate between.

This is because the draw in the gpu doesnt occur imediately it is issued in the cpu.

Very interesting, thanks.