To be fair a lot of that sentiment goes back to 286/368 days when there was 64KB memory segments, 640KB DOS limits and stuff like that some of which was IBM's fault, some Microsoft's but a lot of the blame for that mess was Intel's. Although some of it might just be Mac fanboyism of course...
My take on x86 is that the performance is there now despite of x86. By which I mean that because of all the resoruces which Intel (& AMD) threw at improving x86 over the years, x86 has become the performance leader. But if you were to go back to the mid 80s and look at what was available then (80286, 68020, MIPS, ARM, SPARC etc.) and predict which one would win the performance war, x86 would probably have been an outsider. Sure, if thinking in terms of 'money = resources to win the race', the fact that Intel had won the IBM PC contract might have been the most relevant factor.
Point being: if the same resources had been thrown at any of the alternatives to Intel (or even some Intel RISC design like their DSPs), I am sure that not only would those alternatives have the performance crown now but their performance might be higher since x86 was a poor place to start compared to the others with their flat memory model, lots of registers etc.
X86 was weak up until the Pentium Pro's release in what, 1996? So yes, x86 sucked in the 80s and early 90s.
X86 was designed for a world where the cost of memory far outstripped everything else. It is still the most memory efficient design on the common market, even though it's not a concern anymore.
Most RISC processors were designed for a world where die size was expensive. This is also no longer true, but ARM processors can generally do more with less die size and thus go into the lowest cost devices.
ARM co-opted the low power market, and they have some innate advantage there. X86 has some innate advantages in the high performance market.
X86 needed more research/money to gain some of the performance features innate to RISC designs, and because of its inefficiency in die space, it took longer for important hardware components to be integrated on die.
That said, despite dumping out of order execution, the Intel Atom still performs up there with the best performing RISC designs that lack OOE as well, and competes with low end OOE designs (ie, ARM and AMD's brazos).
Most of the burden of writing x86 code was absorbed into compilers. Thankfully, decades of research and a decoupling of the logical representation from the machine code representation has made x86 compilers quite good.