I haven't watched that video completely, but I see that it follows Baraff's literature to solve for constraint *forces* (impulses / time_step). He uses essentially the same method as found in the constraint-based literature for non-penetration constraints using the LCP formulation. However, he uses quadratic programming (a global solver) to solve for the multipliers.
As Dirk said, you'll need to define your gradients (normals from the SAT for the collision case), each of which will be inserted into a Jacobian matrix for a general constraint-based system. From there you compute the lagrange multipliers as showed in Catto's slides. I think it is important to understand constraints geometrically when implementing these things. So yeah, we do look for a collision normal and point and only then we can find the force magnitude by relating it to the impulse-momentum law in Euler form and then we find the new relative velocities of each body after the collision/contact.
BTW, different authors use the term impulse and force interchangeble, but assuming the force is constant during the time step then impulse = force * time_step. Therefore the lagrange multipliers both refer to the same thing.
(Do not take this too theoretical but here is an example without using Catto's simplification to solve for the multipliers (impulses) for a distance constraint.)