If you are using a 32 bit OS, you will not be able to use all 4Gb, especially true with the MS implementation of hardware address allocation. edit, without PAE that is.
IE, with Windows XP Pro, you will probably see around 3Gb usable RAM when you put 4Gb in.
64 bit, no problem.