Your issue is certainly the 32-bit software you're running.
However, if your calculations are starting to get that intensive, I'd really recommend moving away from Excel for your heavy lifting. I recently ran into issues with extremely large data sets stored in an Excel workbook. Just to load...