First and foremost, the rumor stipulates that the change wouldn't happen soon. Not in 2011, not in 2012. 2013 is a maybe. A lot can happen to ARM architecture in 3 years. Compare the performance of late-2008 T-Mobile G1 with the late-2011 tablet/phone running a Tegra Kal-El (quad-core Cortex-A9).
Of course, Intel and AMD won't be standing still either. But look at the direction that Apple is increasingly moving towards. Lower power, longer battery life, thinner, lighter, solid state storage (spinning platters/optical discs are going extinct in Cupertino). Apple secretly wants all their laptops to be as thin as the MacBook Air, and all their desktops to be the size of a hockey-puck. Power users be damned. This is a post-PC world now, and Apple is making bank off casual users (just look at iPad sales).
Above all else, Apple is in love with being as vertically integrated as possible. If they can ditch Intel and shove an Apple A32 CPU inside their MacBooks, they will. iOS and OSX are slowly blurring together, and iOS already runs like butter on a lowly Cortex-A8.