I upgraded from a 2500 to a 6700k, mainly because my sata3 ports had died on my old motherboard. my 2500 was running at 3.8ghz turbo, and the 6700k is currently at 4.5ghz. There is definitely a noticable difference, mostly in minimum framerates, though average/max didnt change a ton. The biggest change is the platform upgrade, now having all sata3 ports, M.2 for a future upgrade, usb 3.1 type-A and C ports, better onboard sound, etc...
In the end it all depends on the games you play. Some are more dependent on CPU, some on GPU. I think a lot of the performance increases i got were going from a non-K to a K processor and overclocking it gaining an extra 700mhz raw speed. Throw in the 4 generations of 5-10% IPC improvement and to me, it was a worthwhile investment.
The biggest difference likely came from the higher clock speed. Had the 2500K been OC'd to 4.5 GHz (very doable on SB), it would have been much closer. If you had a 2600K rather than a 2500K, the addition of HT would have put it within spitting distance of the 6700K.