I'd say the things most likely to go out from overclocking, in order
1). Graphics card
2). Ram
3). anything else
Graphics card first, because I've experinced this first hand, and because it's harder to keep these cool. I wanted performance so I pushed 75C at 400/1100 on my 6800GT and now I'm stuck at 6800GT speeds and 87C in WoW at times. It still works fortunately.
Ram second, because you can push it to a point but after that it'll just be too hot and will burn out on you.
CPU third and definitely least because of tech like Cool 'n' quiet, and because it seems easier to keep a CPU cool anyhow.