Well you have to remember that Apple has full control over the hardware and software stack for quicker time to market, and has a huge normalized SoC die size and transistor budget too. So for Tegra K1 Denver to compete with A8X (if not exceed A8X in some areas) in single-threaded CPU performance and GPU performance (with a more advanced and forward-looking GPU feature set too) is a great accomplishment. That said, NVIDIA, Qualcomm, Intel, etc. are only indirect competitors to Apple, because they provide SoC hardware for open ecosystems such as Android and/or Windows tablets and phones.
Note that with a 28nm fab. process, it was not really feasible to add more than two Denver CPU cores without blowing past SoC die size and transistor budget. And one has to think that the vast majority of mobile apps cannot take good advantage of more than two CPU cores in the first place (literally every single iOS product other than iPad Air 2 has no more than two CPU cores).
A8X is most certainly not a scaled up version of a phone chip. The GX6650 6-cluster GPU is entirely new and has not and will not be seen in a phone anytime soon (if at all), and a 3-core enhanced Cyclone CPU has not and will not be seen in a phone anytime soon (if at all) either (not to mention the fact that the A8X memory interface is wider and more power hungry than A8 too). On the other hand, Tegra K1 Denver is expected to make it's way into a high end smartphone(s) and is said to be less power hungry and more power efficient than the Tegra K1 Cortex variant.