The electromagnetic force (along with the gravitational force, not sure about the other two forces but since they are so close range I guess it doesn't matter) still propagates at the speed of light, so you're still going to get some amount of latency. Just because there's no physical connection between point A and point B doesn't mean you'll be able to instantaneously transmit information between them, even if they are very close in space.
And, no, that's not how quantum computers work.