i've wrote a small vertex shader for test only and i THINK it should work :P the real question is how can i get the output of the shader when i return to c++? i mean: during the render cycle i call the shader, it does something, then i return to normal code..and now? where can i take the output result? how can i obtain the values contained in oPos (for example) ?

I'm very curious if anyone has any ideas on how to get those vertices out as well myself. As far as I know, you can't, you have to get the data out as rendered pixels...

so anyone have an idea on how to actually get the vertex shader output data?

You don't get the outputs of the vertex program. They are sent to the fragment program if any or the "default fragment program" (the stuff that draw dumb triangles with textures/colors/smoothing etc.).

The common method to debug gpu programs is to modify them to write the data you are interested in to the colour register. I.e. in order to check the normals you write the normal to the colour register (you'll need to scale and bias it because of clamping) and visually interpret the result (i.e. no red = normal pointing to left, bright red = normal pointing to right, etc.).

In addition to the above you can render to a texture and then read back the value so that you can display them textually. When using this method to check vertex programs you probably want to be rendering in GL_POINTS mode.

Enigma

the problem is that i have to compute light position for ppl in the render cycle and i would like to do this with a vertex shader..
so i got a lightToVertex vector and a TBN matrix for the i-th vertex of the face i'm currently rendering..all i need to do is to compute the dot product and save the results in another vector i'll use for cube map position. It's all. nothing strange :D

so i've wrote this vertex shader that computes the dot product... but now? how can i store the results in the vector i will use?

Once again: you can't. Results of VP are interpolated over the triangle and passed to current FP. So you will either have to do the stuff on CPU or wait for render-to-vertex-array extensions. In your case just compute object space light position on cpu and pass it to VP as local parameter.

I'm not sure what exactlly you're trying to do but, if all you want to do is implement per-pixel lighting then
you don't need to send the data back to the CPU.

Instead, output the normal and light's direction vector from your vertex shader and then use these in your pixel shader which implements per pixel lighting.

You can use the TEXCOORDn input/output synmantecs to send the
vectors to the pixel shader. Now perform the dot product and whatever other calculations you need in the pixel shader and output the final color result.

Hope that answers your question.

Quote: Original post by b3rs3rk
so i've wrote this vertex shader that computes the dot product... but now?

Hey, now, you just have to write the corresponding fragment program and post a screenshot here ;*)

If you want to check the vertex->light vector, just use this vector to translate all vertices in the vertex shader (be careful about the transform matrices), and use a key to enable/disable the vertex shader. this way, if everything is OK, all vertices will be translated toward the point light when the VP is enabled.

So I guess there's no way to grab the vertex buffer post-shader transformation? I'm assuming this is a driver issue, since vertex buffers are clearly writable from the CPU, and need to be stored at some point in memory for rasterization.

This was what I had assumed at first, but I'm glad someone asked this because I've always wanted to know-and you guys are definitely the people who would know if anyone would.

>So I guess there's no way to grab the vertex buffer post-shader >transformation? I'm assuming this is a driver issue, since >vertex buffers are clearly writable from the CPU, and need to >be stored at some point in memory for rasterization.

No driver issue. The vertex shader transforms the vertices into another buffer in video memory not in system memory. The transformed vertices are redirected into the pixel shader. All is done in video memory.

Anyway... you can use Direct3DDevice9::ProcessVertices() to transform your buffer into other buffer, but the process is done in the CPU not in the GPU so it is just like you make this inside a function in your program.

Luck!
Guimo