There's something to the notion that video killed the radio star. I think it really started going to shit when your looks became so ultra-important. It didn't happen instantly (well, it pretty much did for Christopher Cross, heh), but by the early 90's, most of the older or less attractive acts that made it onto MTV did so because they had some sort of legacy and popularity going back several years.
I also agree with everything in tcsenter's post about piracy and corporate sponsored tours. I think you can add in consolidation of record labels too, with music becoming even more of a product than it already was, sold by publicly-traded corporations that have quarterly earnings to meet. In that environment you don't take a lot of chances, you don't wait around for 3 albums for an artist to "develop," and you pretty much stick to the formula that's working at the moment. Yeah it's always been that way to an extent, but I think it's worse now.
So I think it's been business and technological factors have helped create the larger than normal generation gap that we have right now. The last huge generation gap in music, starting in the early/mid 60's, was more driven by evolution in society. In the span of 4-5 years, pop music went from folk or crooners backed by strings to distorted guitars, heavy backbeats, and screaming & wailing. You pretty much were in one camp or the other on that!