I want to add friction, static and dynamic, to my resting contact force resolution. I formulated the problem as an LCP, but in some situations it seems the LCP is unsolvable. As a simple example consider the, badly drawn, diagram below.
[attachment=30505:forces.jpg]
The horizontal box has a very high mass and is initially stationary. I will largely ignore the forces on this box because its mass is assumed to be so big that the forces involved are unable to accelerate it.
The vertical box isn't quite vertical, but the lower left vertex is in contact with the horizontal box and slightly to the right of the vertical box's centre of mass. The vertical box is moving to the right and has a force acting downwards on it. The reactive contact force would push up on the vertical box and this would also cause a frictional force to the left. The vertical box's centre of mass is high above the contact point so the friction will cause a large torque. The reactive force will cause very little torque. This can actually leave us in the situation where applying a larger reactive force leads to the boxes accelerating more towards each other.
I thought about this situation and what would happen in the real world. I think there would be a very large reactive and frictional force applied over a short period of time that would stop the vertical box moving to the right. I could probably detect and handle this situation by setting tangent velocities to 0.
I also thought that maybe I could approximate and calculate friction based on the forces acting at the start of each frame. Then resolve the contact forces, removing any effect that friction had on the normal acceleration.
How are people practically handling friction in their physics engines? I would like my physics engine to run in real time and the second approach simplifies things quite a bit, but is more approximate. Which of these approaches would make the most sense to you or do you have a better idea?