If you use DX12 with commmand queues and use fences I dont know thats its a particular problem. Note this though:

• Don’t toggle between compute and graphics on the same command queue more than absolutely necessary
• This is still a heavyweight switch to make

From https://developer.nvidia.com/dx12-dos-and-donts. Perhaps that is what you are meaning?

If you mean DX 11, please say.

Switching between compute and graphics on Nvidia hardware will cause a "wait for idle" since they won't do both simultaneously unless you use multiple command queues. Whether or not that actually matters for your performance depends on what you're doing. If you're already requiring some kind of barrier/sync/flush between your draws and dispatched, then it probably won't matter.