I implemented a simple AABB and Circle collision system in my engine. It mostly works ok but sometimes I get really weird effects.
When a Circle touches (collides) AABB, sometimes it doesn't bounce right away but it enters into the box. It's barely noticeable and when it enters ti doesn't automaticaly go out but it wiggles for bit and after 1-5s it exists in correct direction.

This is my code to change direction after Circle collided with AABB (collision returned true):

if (psprite->Tag() == TagType::WALL_LEFT)
{
	dir = Vector2(-dir.x, dir.y);
}

Where psprite is a Wall sprite and this code is located in Ball class (Circle collider).
Have you encountered something similar? I just can't figure what's wrong.

You'll have to show a little bit more of you collision detection logic, but usually this kind of thing is related to numerical instability and/or quickly moving objects.

If your circle is moving fast enough, it may penetrate far enough on the first colliding tick that even through the direction is reversed, it's still marked as colliding on the next frame, causing the direction to be flipped again, and so on... 

There are several relatively easy fixes:

• You can not only flip the direction, but also move the circle in the new direction just until it is no longer intersecting the wall. This will prevent the collision being registered a second time.
• Rather than flipping the direction, you can always set it to be away from the wall. This will make the collision being registered multiple times harmless.
• You could modify your collision detection to track object collisions until they are no longer in contact, and then use that information to only flip the direction when they first make contact.

Yes that's it. That's why the object is wobbling when is in another object. Because collision fires each time and every time it changes its direction by -dir.

I'll try to implement one of these methods you mentioned.
Thanks.