Here is how continuous physics works in a nutshell. We start with the general discrete solver step:
1) You integrate external forces (e.g. gravity)
2) You solve the joint and contact constraints
3) You integrate the velocities to find the new positions
Of course we might have tunneled right now. So in a continuous solver the new positions are tentative and we now enter the continuous part of the solver:
1) For each shape you build the swept AABB and add it into the broadphase.
2) For each contact you compute and cache the TOI
3) You advance the bodies to the minimum TOI, build the contacts and solve them
4) You invalidate the cached TOI for the involved bodies and recompute them
5) Repeat from 3) until you have reached the end of the timestep
I left out a lot of detail, but this is the general idea. Note that you still need to run the normal contact phase before entering the continuous phase.
So to answer you question. Yes, of course you need to compute the contact at each TOI.
HTH,
-Dirk