If you dont need backward compatibilty (eg, with some older addins or supporting apps), then definitely use 64bit MS Office given a 64bit compatible machine.
The narrative is that 64bit programs, in generlal, will exhibit better performance & memory uses, particulary when involving large data sets and/or using apps involving complex calculations.
For me personally, the 32bit versions are fine as I like the simpler retro look often seen with 32 bit (legacy) versions. Ive never really noticed a performance improvement (over 32bit), plus bothersome, is that I've not been able to glean numerical precision increase with, for example, EXCEL. (You can see such issue when trying to use EXCEL matrix inversion.)
Jus Say'n