How long do you consider a long period?
The oxide layer on the foil starts to degrade, which can result in leakage, causing some degree of additional heat and loss of capacitance. This usually doesn't reach a problematic level until several years have passed.
The layer will reform once power is applied again, but if the leakage is excessive, it can cause the capacitor to rupture, so ideally (not that most people would actually do this) a very low voltage is applied and then ramped up over several hours to reform them.
For practical purposes, if I'm going to deploy a used PSU that's been sitting around unused for years I'll just hook up an old HDD (or I have a load tester but usually don't bother using it), jumper the PS_On signal line to a ground contact on the connector, and let that run a few hours before powering a system, and I'll have opened the PSU, made sure there was no dust buildup, examine the capacitors for bulging, and lube the fan if it has a sleeve bearing.
Then there are lower quality capacitors with an unstable electrolyte, commonly the less expensive chinese brands, will break down and produce gas and vent, but this happens if they are kept in use too, typically at an even faster rate.
Finally there is the longest term issue of the electrolyte drying out, but usually this takes a few decades to cause enough degradation to matter, although the demands of the circuit determine how far out of spec a capacitor can get before it poses a problem.
A good quality PSU will have capacitors over-spec enough that you should expect well over a decade of service life whether powered or not, if it is not used at high temperature or near full rated output, which also heats the capacitors due to their ESR, causing them to create a little heat during normal use, even if in perfect condition. An old rule of thumb was that for every 10C rise in capacitor (not necessarily ambient...) temperature, the lifespan is cut in half. This is only for electrolytic, not any of the solid capacitors which generally do not have nearly as much lifespan reduction within their rated temp range. There are more variables involved, a single post can't address everything.