USB devices may be more convenient. And I assume if they're class compliant they can be used with or without special software which may or may not do "awesome stuff". USB devices include their own DAC/ADC & amp circuitry, which means you don't need to use your own "sound card" or "onboard" sound. USB ports are often "noisy" with regards to power delivery, which introduces a "hum" you may or may nor hear or care about (my motherboard for example came with a special "DAC" usb port to eliminate this problem).
"Normal" analog 3.5mm TR/TRS plugs go into a "sound card" you already have, which if you spent any decent amount of money on (or if your "onboard" sound is good) may well have much better DAC/ADC & amp circuitry. Without a USB means you're not also paying for redundant hardware if you already have a decent "sound card" or "onboard" sound. Bundled software with "analog" headset may or may not have as much "awesome stuff". Lots of "onboard" and "internal" "sound cards" suffer from "noise" from power deliver or EMI inside the case, leading to a "hum" or background "noises" you may or may not hear or care about.
Edit: fix some speling