It probably defaults to a lower speed when you fill all 4 slots... motherboards have a tendency to do that, set it manually to PC-6400 in BIOS and make sure it runs stably.
Also, if you're not overclocking, then you wont be able to tell the difference anyway, because your FSB is less than 1600Mhz