Hi guys,

I've been struggling to create a function to draw an arrow with 2 lines making up the arrowhead.
The challenge is that I somehow can't get the rotation right.

I've pasted the code below and an image of the result.
Findings so far, when I replace the 3x3 part of the transform matrix to identity, I get the same visual result.
Also when I switch colums A and C in the matrix, this specific arrow looks good (with is pointing in the positive X direction).

Any input would be appreciated.

bool CDebugDraw::AddArrow(const CR_VECTOR3 &pPos1, const CR_VECTOR3 &pPos2, const CR_VECTOR3 &pColor, const bool pDepthEnabled, const bool pPersistent, const uint64_t pLifeTime)
{
	if(!AddLine(pPos1, pPos2, pColor, pDepthEnabled, pPersistent, pLifeTime)) return false;

	/*
						p1						------- X
					  /	| \						|
					 /	|  \					|
				    /	|   \					|
			    p2 -----|-----p3				| Z
						|
						|
						|
						|
						p0

	*/
	CR_VECTOR3 arrowDir = pPos2 - pPos1;
	arrowDir.Normalize();

	// model space
	CR_VECTOR3 p1 = CR_VECTOR3(0.0f, 0.0f, 0.0f);
	CR_VECTOR3 p2 = CR_VECTOR3(0.2f, 0.0f, -0.2f);
	CR_VECTOR3 p3 = CR_VECTOR3(-0.2f, 0.0f, -0.2f);

	// transformation: translate and rotate
	CR_VECTOR3 transl = pPos2;

	CR_VECTOR3 colA = arrowDir;
	CR_VECTOR3 tVec;
	if(colA.x != 0 && colA.z != 0) tVec = CR_VECTOR3(0.0f, 1.0f, 0.0);
	else tVec = CR_VECTOR3(0.0f, 0.0f, 1.0f);

	CR_VECTOR3 colB = CMathHelper::CrossVec3(colA, tVec);
	CR_VECTOR3 colC = CMathHelper::CrossVec3(colB, colA);

	CR_MATRIX4X4 transform;
	transform.m11 = colA.x;		transform.m12 = colB.x;		transform.m13 = colC.x;		transform.m14 = 0.0f;
	transform.m21 = colA.y;		transform.m22 = colB.y;		transform.m23 = colC.y;		transform.m24 = 0.0f;
	transform.m31 = colA.z;		transform.m32 = colB.z;		transform.m33 = colC.z;		transform.m34 = 0.0f;
	transform.m41 = transl.x;	transform.m42 = transl.y;	transform.m43 = transl.z;	transform.m44 = 1.0f;
//	transform = CMathHelper::ComposeWorldMatrix(transform, CR_VECTOR3(1.0f), CR_VECTOR3(0.0f, 90.0f, 0.0f), pPos2);

	// transform to worldspace
	p1 = CMathHelper::TransformVec3Coord(p1, transform);
	p2 = CMathHelper::TransformVec3Coord(p2, transform);
	p3 = CMathHelper::TransformVec3Coord(p3, transform);

	if(!AddLine(p1, p2, CR_VECTOR3(1.0f, 0.0f, 0.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;
	if(!AddLine(p1, p3, CR_VECTOR3(1.0f, 0.0f, 0.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;

	if(!AddCross(p2, 0.02f, CR_VECTOR3(0.0f, 0.0f, 1.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;
	if(!AddCross(p3, 0.02f, CR_VECTOR3(0.0f, 0.0f, 1.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;
	return true;
}

Incorrect result:

Aimed/ correct result (independent of arrow direction):

You need to invert the matrix you're building in order to transform p1, p2 & p3 from model space to world space.

Okay I think I have a solution cozzie. This works on paper but I didn't actually "simulate" it to verify that it works.

First, let's actually forget about the arrow for a second and also forget about "rotation" and "translation" and "scale". Try to focus on what this problem is really about: It's a change of basis.

You have a set of points that exist in the standard basis for R^3 (aka "model space"):

You are looking for a matrix M : R^3 -> R^3 which maps the standard basis to a new basis B:

Given points p0 and p1 of the arrow in world space, you can calculate the new basis vectors:

The new "Z" basis is the arrow's direction: p1-p0.  The new "X" basis can be obtained via the cross product with the world's "UP" direction.  The new "Y" basis can be calculated with the cross product of the two new basis vectors.

Now, because there is also a translation involved, it is necessary to first translate to (0,0,0), apply matrix M, then translate back. Because the standard basis is already at (0,0,0) it makes things quite a bit easier. The matrix M is simply:

In your code that would look something like this:

bool CDebugDraw::AddArrow(const CR_VECTOR3 &pPos1, const CR_VECTOR3 &pPos2, const CR_VECTOR3 &pColor, const bool pDepthEnabled, const bool pPersistent, const uint64_t pLifeTime)
{
    if(!AddLine(pPos1, pPos2, pColor, pDepthEnabled, pPersistent, pLifeTime)) return false;

    /*
                        p1                       ------- X
                      / | \                     |
                     /  |  \                    |
                    /   |   \                   |
                p2 -----|-----p3                | Z
                        |
                        |
                        |
                        |
                        p0

    */
    
    // Form new basis vectors
    CR_VECTOR3 e3 = pPos2 - pPos1;
    CR_VECTOR3 e1 = CMathHelper::CrossVec3(e3, CR_VECTOR3(0, 1, 0));
    CR_VECTOR3 e2 = CMathHelper::CrossVec3(e1, e3);
    
    // Fill in basis vectors into our "B" matrix (note it is transposed because your library uses row major)
    CR_MATRIX4X4 M;
    B.m11 = e1.x;    B.m12 = e1.y;    B.m13 = e1.z;    B.m14 = 0;
    B.m21 = e2.x;    B.m22 = e2.y;    B.m23 = e2.z;    B.m24 = 0;
    B.m31 = e3.x;    B.m32 = e3.y;    B.m33 = e3.z;    B.m34 = 0;
    B.m41 = 0;       B.m42 = 0;       B.m43 = 0;       B.m44 = 0;
    
    // Create translation matrix "T" (note it is transposed because your library uses row major)
    CR_MATRIX4X4 T;
    T.m11 = 1;       M.m12 = 0;       M.m13 = 0;       M.m14 = 0;
    T.m11 = 0;       M.m12 = 1;       M.m13 = 0;       M.m14 = 0;
    T.m11 = 0;       M.m12 = 0;       M.m13 = 1;       M.m14 = 0;
    T.m11 = pPos1.x; M.m12 = pPos1.y; M.m13 = pPos1.z; M.m14 = pPos1.1;
    
    // Combine both matrices into the final transformation matrix "M"
    CR_MATRIX4x4 M = CMathHelper::MultiplyMatrix4x4(T, B);
    
    // Define the arrow vertices in model space
    CR_VECTOR3 p1 = CR_VECTOR3( 0,   0, -1);
    CR_VECTOR3 p2 = CR_VECTOR3(-0.1, 0, -0.9);
    CR_VECTOR3 p3 = CR_VECTOR3( 0.1, 0, -0.9);
    
    // Do transformation
    p1 = CMathHelper::TransformVec3Coord(p1, M);
    p2 = CMathHelper::TransformVec3Coord(p2, M);
    p3 = CMathHelper::TransformVec3Coord(p3, M);

    if(!AddLine(pPos1, pPos2, pColor, pDepthEnabled, pPersistent, pLifeTime)) return false;
    if(!AddLine(p1, p2, CR_VECTOR3(1.0f, 0.0f, 0.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;
    if(!AddLine(p1, p3, CR_VECTOR3(1.0f, 0.0f, 0.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;

    if(!AddCross(p2, 0.02f, CR_VECTOR3(0.0f, 0.0f, 1.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;
    if(!AddCross(p3, 0.02f, CR_VECTOR3(0.0f, 0.0f, 1.0f), pDepthEnabled, pPersistent, pLifeTime)) return false;
    return true;
}

Thanks, with this input and a lot of chat on the Discord GD, I almost solved it.
After some addition hour(s) of debugging the last issue seemed to be the order of matrix multiplication (Basis vectors and translation). All working like a charm now.

For future reference/ others, here's the fixed code:

bool CDebugDraw::AddArrow(const CR_VECTOR3 &pPos1, const CR_VECTOR3 &pPos2, const CR_VECTOR3 &pColor, const bool pDepthEnabled, const bool pPersistent, const uint64_t pLifeTime)
{
	if(!AddLine(pPos1, pPos2, pColor, pDepthEnabled, pPersistent, pLifeTime)) return false;

	/*
						p1
					  /	| \	
					 /	|  \
				    /	|   \
			    p2 -----|-----p3
						|
						|
						|
						|
						p0

	*/

	// form new base vectors
	CR_VECTOR3 e3 = pPos2 - pPos1;
	e3.Normalize();
	CR_VECTOR3 e1 = CMathHelper::CrossVec3(e3, CR_VECTOR3(0.0f, 1.0f, 0.0f));				// how about arrow direction 0 1 0 ?
	e1.Normalize();
	CR_VECTOR3 e2 = CMathHelper::CrossVec3(e1, e3);
	
	// fill basis vectors into our 'B' matrix	(transposed to work with Mathhelper/DXmath)
	CR_MATRIX4X4 bMat = CR_MATRIX4X4(CR_VECTOR4(e1, 0.0f), CR_VECTOR4(e2, 0.0f), CR_VECTOR4(e3, 0.0f), CR_VECTOR4(0.0f, 0.0f, 0.0f, 1.0f));	

	// translation matrix
	CR_MATRIX4X4 transMat;
	transMat.m41 = pPos2.x;			transMat.m42 = pPos2.y;			transMat.m43 = pPos2.z;			transMat.m44 = 1.0f;

	// multiply B and translation
	CR_MATRIX4X4 finalMat;
	finalMat = CMathHelper::MatrixMultiply(bMat, transMat);

	// model space arrowhead lines
	CR_VECTOR3 p1 = CR_VECTOR3(0.0f, 0.0f, 0.0f);
	CR_VECTOR3 p2 = CR_VECTOR3(-0.1f, 0.0f, -0.1f);
	CR_VECTOR3 p3 = CR_VECTOR3(0.1f, 0.0f, -0.1f);

	// transform them
	p1 = CMathHelper::TransformVec3Coord(p1, finalMat);
	p2 = CMathHelper::TransformVec3Coord(p2, finalMat);
	p3 = CMathHelper::TransformVec3Coord(p3, finalMat);

	// draw lines
	if(!AddLine(p1, p2, pColor, pDepthEnabled, pPersistent, pLifeTime)) return false;
	if(!AddLine(p1, p3, pColor, pDepthEnabled, pPersistent, pLifeTime)) return false;
	
	return true;
}