I want to relinquish the GPU from the windows 7 host and give it to the Ubuntu 16.04 VirtualBox VM, I know there is something called the GPU passthrough. But I only found some videos on how to passing from Ubuntu host to windows guest. I want to do the opposite which is passing the GPU from the windows host to Ubuntu guest.

Thanks

Jack

Unfortunately this isn't a supported feature on a windows machine using any virtualisation platform i've seen. Virtual Box, VMWare Workstation, Hyper-V, while each have their own versions of PCI pass-though, none of them allow you to do GPU specific pass-though.

You could probably achieve this behavior with a type-1 hypervisor, in that you install VMwares ESXi, or another Hypervisor (KVM/XEN) on a bare-metal box, and deploy VMs on top of that, as opposed to the consumer based type-2 "run on your desktop" hypervisors 

Depending on the GPU that you have, you should be able to achieve GPU Passthrough if you choose to go the type-1 hypervisor route. If you have a Geforce card, def do your research, Nvidia goes through great lengths at the driver layer to coerce you to buy into their Quadro line when it comes to passthrough. 

Some relevant threads/articles:

https://elatov.github.io/2018/04/esxi-65-passthrough-video-card-to-plex-vm

https://www.reddit.com/r/nvidia/comments/5m07h2/pci_passthrough_in_esxi_for_consumer_nvidia_cards/

https://petr.io/en/blog/2017/12/26/running-nvidia-cards-vmware-esxi/