Do you mean that you want to actually have the actuals applications background be transparent, so that you can see whatever application/desktop is behind it? Including being able to click through those transparent parts? If yes, then you'd need to use the dwmapi (https://docs.microsoft.com/en-us/windows/win32/api/dwmapi/) and functionality like DwmExtendFrameIntoClientArea.
Otherwise, I'm not sure what your problem is; mainly since we don not know which version of DX you are even targeting. You can create a render-target with a format that includes an alpha-channel; and depending on your version you can configure the blending-behaviour for the alpha-channel separately from the color. But without the proper version and more details its hard to give you pointers.