hello i use glm and i have modify an example of arcball camera.
ab_start is a vector calculated when i press and ab_curr is a vector in the drag phase

all this two vectos are unprojected and At the end this is the part that don't understand:

   // use a dot product to get the angle between them
        // use a cross product to get the vector to rotate around
        float cos2a = glm::dot(ab_start, ab_curr);
        float angle = glm::degrees(glm::acos(cos2a));
        float sina = sqrt((1.0 - cos2a)*0.5);
        float cosa = sqrt((1.0 + cos2a)*0.5);
        glm::vec3 cross = glm::normalize(glm::cross(ab_start, ab_curr))*sina;


        ab_next *= glm::cross(ab_next, glm::rotate(ab_next,cosa,cross));

i have the cross ,the angle and i do a quaternion axis rotation, now in ab_next i have an orientation, is correct?
But, how i get the view matrix from this orientation?
I have already a view matrix , i calculated it in the costructor of my camera whith the glm::lookat ....

i use the shaders and before draw an object in the shader, i get from the camera the matrix :

glm::mat4 CCameraEx::getMVP()
{
    
    return getPerspectiveMatrix() *mViewmatrix* glm::inverse(glm::toMat4(ab_next));
}

but is wrong.
how i calculate the viewmatrix?
Thanks.

sorry for my english.

Convert the quaternion to standard 3x3 rotation matrix, then you will need the translation column (x,y,z), with considering of "rotate around point", so the final 3x4 matrix should be

3x3rotationpartfromquaternion, and the fourth column vector of translation should be (x,y,z)

x=dot(row1,eyeposition)

y=dot(row2,eyeposition)

z=dot(row3,eyeposition)

where row1/2/3 are 3d vectors of 3x3 rotation part, and eyepostion is just world postion of your camera.

very thanks , but at the end this is the new viewmatrix?

very thanks , but at the end this is the new viewmatrix?

Yes, well you define the view matrix by a normalized 4d quaternion and a 3d position, this information is enough to define the 4x3 view matrix as described in my post, make it a 4x4 matrix by adding (0,0,0,1) fourth row if needed.

All you need is to construct a correct normalized quaternion, then there is a standard method to acquire the 3x3 rotation matrix from it.

Sorry, but I do not think that the shown code in the OP is correct.

a) An axis-angle rotation is another rotation representation than a quaternion. The glm::rotate function expects an angle (2nd argument) and axis (3rd argument), but not a quaternion. (It is in general a mistake to try to understand a quaternion in geometric terms like angle and axis.)

b) The line

float cos2a = glm::dot(ab_start, ab_curr);

computes the cosine between the two vectors weighted by the product of the lengths of the 2 vectors, accordingly to

a . b = |a| * |b| * cos( <a,b> )

Your code line uses the equivalent form

cos2a := cos( <a,b> ) = ( a . b ) / ( |a| * |b| )

Now, the name cos2a is mistakable. Does it mean cos²(a) or cos(2*a) or what? It is further not obvious from our code snippet whether or not the 2 vectors are normalized. I assume that they are and that you actually want to compute

cos_ab := cos( <a,b> ) = ( a . b ) w/ |a| = |b| = 1

c) The code line

float angle = glm::degrees(glm::acos(cos2a));

transforms the angle between the 2 vectors from radiant to degree. However, the result is ignored in the following, although it would be an expected 2nd argument for glm::rotate.

d) The code line

float sina = sqrt((1.0 - cos2a)*0.5);

… well, I thought the basic formula were the trigonometric Pythagoras

sin²(a) + cos²(a) = 1

but then one gets

sin(a) = sqrt( 1 - cos²(a) )

This could mean that the name cos2a actually means the square cosine, but, as already written, those variable does not store the square value. But even then, where does the factor 0.5 comes from?

e) The code line

float cosa = sqrt((1.0 + cos2a)*0.5);

is even more nebulous to me.

f) The code line

glm::vec3 cross = glm::normalize(glm::cross(ab_start, ab_curr))*sina;

computes a vector that stores the imaginary parts of a quaternion (assuming that sina is calculated correctly as the sine of the angle between the 2 vectors). Naming this vector "cross" is misleading, though.

g) The code line

ab_next *= glm::cross(ab_next, glm::rotate(ab_next,cosa,cross));

does something I'm not sure about. First of, the invocation of glm::rotate uses wrong arguments. The function in question is IMO those documented as "Rotate a three dimensional vector around an axis." Then its 2nd parameter need to be an angle in degree (e.g. your variable "angle" would fit), but you use a cosine of an angle (the real part of a quaternion, by the way)!? And the 3rd parameter should be an axis but you put in the imaginary parts of a quaternion!?

When the parameters of glm::rotate would be correct, then the vector ab_next would be rotated analogously to the rotation from ab_start to ab_curr simply due to glm::rotate. What purpose has the remaining of that code line?

thanks, i change to :

		float cos2a = glm::dot(ab_start, ab_curr);
		float angle = glm::degrees(glm::acos(cos2a));

		glm::vec3 cross = glm::normalize(glm::cross(ab_start, ab_curr));


		ab_next = glm::normalize(glm::rotate(ab_next,angle,cross));

		ab_start = ab_curr;

and for explain i get the ab_start like:

glm::vec3 CCameraEx::sphere_coords(GLdouble mx, GLdouble my)
{
	GLdouble ax, ay, az;
	glm::mat4x4 mt = glm::mat4x4(1) * mVV;
	glm::vec3 m = glm::unProject(glm::vec3(mx, my,0.0),mt , mP, glm::vec4(0, 0, 800.0f, 600.0f));
	m = m - ab_eye;

	// mouse position represents ray: eye + t*m
	// intersecting with a sphere centered at the origin
	GLfloat a = glm::dot(m,m);
	GLfloat b = glm::dot(ab_eye,m);
	GLfloat root = (b*b) - a*(ab_zoom2 - ab_sphere2);
	if (root <= 0) return edge_coords(m);
	GLfloat t = (0.0 - b - sqrt(root)) / a;
	
	return glm::normalize((ab_eye + (m*t)));

}

// find the intersection with the plane through the visible edge
glm::vec3 CCameraEx::edge_coords(glm::vec3 m)
{
	// find the intersection of the edge plane and the ray
	float t = ((ab_edge - ab_zoom) / (glm::dot<float>(ab_eyedir , m)));
	glm::vec3 a = ab_eye + (m*t);
	// find the direction of the eye-axis from that point
	// along the edge plane
	glm::vec3 c = (ab_eyedir * ab_edge) - a;

	// find the intersection of the sphere with the ray going from
	// the plane outside the sphere toward the eye-axis.
	float ac = glm::dot(a,c);
	float c2 = glm::dot(c,c);
	float q = (0.0 - ac - glm::sqrt(ac*ac - c2*((glm::dot(a,a)) - ab_sphere2))) / c2;

	return glm::normalize((a + (c*q)));
}

where mVV is the view matrix

nobody?

nobody?

nobody?

nobody?

a precisation : ab_next is a quaternion

I sorry for my hurry, i stop and thinks, i have this code:

ab_next is a quaternion

ab_next *= glm::normalize(glm::angleAxis((angle), glm::vec3(1.0,0.0,0.0)));

.

.
//this is called before draw the object and i'm set it to the prograsm shader like a matrix
glm::mat3 mat = glm::toMat3(glm::normalize(ab_next));
    
    glm::vec3 x1 = glm::vec3(mat[0].x,mat[0].y,mat[0].z);
    glm::vec3 y1 = glm::vec3(mat[1].x, mat[1].y, mat[1].z);
    glm::vec3 z1 = glm::vec3(mat[2].x, mat[2].y, mat[2].z);
        
    float x = glm::dot(x1, eye);
    float y= glm::dot(y1, eye);
    float z = glm::dot(z1, eye);

    glm::mat4x4 mview;
    mview[0].x = mat[0].x;
    mview[0].y = mat[0].y;
    mview[0].z = mat[0].z;
    mview[1].x = mat[1].x;
    mview[1].y = mat[1].y;
    mview[1].z = mat[1].z;
    mview[2].x = mat[2].x;
    mview[2].y = mat[2].y;
    mview[2].z = mat[2].z;
    mview[3].x = x;
    mview[3].y = y;
    mview[3].z = z;
    mview[3].w = 1;
    mVV = mview;
    return getPerspectiveMatrix() * mview;

the object rotates around a big sphere and i would that the object rotating around itself.
the object is at 10,5 of distance in the z axis.
I don't know where comes this big sphere.

thanks