Your original question was about how to figure out if the target was within a field of view defined as a threshold angle. Using the dot product is the natural solution to that problem, and it should work just fine.

Now you show us some code and it turns out that you are trying to use the dot product to decide whether to rotate left or right. This makes no sense.

We will only be able to offer good advice if you define your problem precisely. Now, what exactly is the behavior you want?

Well what I want is an object to face/rotate towards a target point. Also I want to know how close it is to facing the right direction(To determine accuracy).

If you want to know which way to rotate, take the shipFacingVector, rotate it 90 degrees (by mapping (x,y) to (-y,x)), and now take the dot product with the shipToEnemy vector. The sign will tell you if you need to rotate to the right or to the left.

Of course if you simply do this, you'll get oscillations when you are already facing the right way. But you can worry about that once the code is doing roughly the right thing.

Indeed. That is effectively what the short code snippet I put towards the end of my last post here does, albeit as a result of explaining the 3D case and then simplifying for 2D.

Or you can do it with atan2; here is some Flash code for solving a similar problem (Flash is inherently 2D so I didn't bother with a general solution):


Edit: I also realised that doing d.l only works if the object is not behind you, so you should do a preliminary check that d.f is within the threshold (cosine of the maximum view angle) anyway before calculating the angle by d.l.