I noticed I had 1-2 logical cores pegged at all times by MMLoadDrv.exe and looked this up to find it was ATI HDMI audio drivers.
Looking through the event log also revealed about a dozen various error messages per second from ACEEventLogSource. It definitely wasn't doing this before a few months ago, so I'm thinking recently updating to IE11 and the new DirectX may have triggered it. I had video drivers from April, which would have predated that release by a bit.
Updating the video drivers to the latest and rebooting solved the issue.
Looking through the event log also revealed about a dozen various error messages per second from ACEEventLogSource. It definitely wasn't doing this before a few months ago, so I'm thinking recently updating to IE11 and the new DirectX may have triggered it. I had video drivers from April, which would have predated that release by a bit.
Updating the video drivers to the latest and rebooting solved the issue.