The devil is in the details. First of all, please remember the prices and availability are different outside the US. Your quoted MSRP prices are useless here. Also, FWIW, I bought the RX570 in December 2019, a few months before the pandemic. So at least in theory it *should* have ironed all the bugs by then.
Second, a card that's just dropped in the machine and instantly works with any game or application you throw at it should be the normal experience. Yet, the RX570 even had trouble displaying 4K content (multiple HDCP issues every day, no matter how many HDMI cables I changed), aside from gaming.
Some thoughts:
I had an rx580, (purchased new) and if I remember right it had a HDMI 2.0 port, not the 2.0a or 2.0b.
So it could do 4k @ 60 Hz, and that was it. 4k @ 60 Hz - HDR? it could be forced, but it would break all the time
Never had HDCP problems, but never played protected content either
Thing is, on the card spec sheet it was clear that it was either 4k @ 60 Hz, or 4k @ 30 Hz with HDR. I wonder if you just pushed the 2017 spec HDMI port beyond its specs.
I did use hardware encoding on my rx580, but it was with discord to share my screen with my friends. Seemed to work just fine. Very different app though.
It was also unable to use hardware encoding in software packages like ClownBD. And that was in 2020, a good 3 years after it became available.
Using GPU encoders for movie backup is very rare. Most people prefer the much superior CPU encoding. From my minimal research, it seems nvenc specifically is disliked for its poor handling of dark areas.
Yes, AMD got better hardware encoding in the next generation, but even then it trailed Nvidia spectacularly. You don't have to take my word for it. See this detailed comparison:
https://obsproject.com/forum/resour...s-2020-nvenc-vs-amf-vs-quicksync-vs-x264.998/
The problem with that, is the OBS people themselves indicate the AMD encoder is not a priority, the
plugin was done years ago by a guy who stated he did not know what he was doing, and they are not going to do anything about it. They do not care.
It seems most serious OBS users are targeting CPU encoding, which kind of explains why they don't care.
If your going to do youtube / twitch streaming on the cheap with an AMD card right now, it looks like ReLive is the best / only option. On the Nvidia side there is geforce experience, although it seems nvidia does offer official OBS plugins.
But it seems from the research I did, the reason most people use OBS is for its CPU streaming capability. Making this entire topic irrelevant.
-----------------------------------
While your experience with your rx570 was horrible, lets be blunt. Your using niche software tools in a manner that is niche even for those tools. Nearly everyone using those software tools with those use cases are using CPU encoding.