Since it started working after you pulled some USB devices, I would say that you either have a bad USB device among the items you disconnected, there is something wrong with the USB port/controller, or your BIOS is set to allow boot from USB and one of your USB devices has the ability to boot the system (USB drive enclosure or thumb drive) with portions of a bootable system installed but not enough to actually boot to an OS.
Another possibility is that your power supply is starting to fail and no longer has enough power to start up all of your internal devices as well as all of the external USB devices you are using that draw power from the USB bus.
Plug each of the devices you removed back in one at a time and try to boot the PC with each individual item connected until you find out which one is causing the problem. If it doesn't happen with just one item connected, try with two, then three (if needed), etc.
If you can determine that it only happens when multiple devices are connected, then I would suspect the power supply.
If it only happens when one specific device is connected, it's a problem with that device.
If it happens when any single device is connected, then it is likely a problem with the USB port/controller itself.