Have you tried via RDP?
Passing through an entire USB controller is kinda overkill. If you really need to do this, what you could do is get a USB sound card and plug the speakers and just pass the USB device, NOT the port, through to the VM.
But I would really just use RDP its much easier...