If Intel extends their fixed function units a bit, over a few generations (less direct mapping of DX, but still fairly simple computational units--something halfway between SB's IGP and current Radeons), they'll have flexibility there to tackle some corner cases better, and the many-core CPU can work well as leverage for branchy code. For all of its ills, Hyperthreading aught to be excellent for handling the small wait times between CPU & GPU, when dealing with fine-grained work.
Combine that with having the expensive many-weak-core devices for the HPC market, and nVidia will end up even more screwed. They're in no short-term danger, IMO, but they're going to have to push down some of that pride and ego, and learn from their competitors.
As part of that they will need to stop with the black boxes (same goes to you, Imagination!). Hide the secret sauce behind well-documented interfaces; because the ability for Linux kernel and Xorg hackers to fix bugs will come to matter more than all of the performance you can bring. Also, I doubt we're all that far away, outside of Windows, at least, for FOSS compilers to do well with GPGPU (when/if Gallium3D becomes stable, and supports a few FOSS-friendly GPUs, the capability will be there to exploit).