Maybe you set PCI-E frequency too high? For example, setting PCIE-E frequency to 117 MHz may help overclocking. but only with a AGP card...
I thought the PCI-E frequency impacted the PCI-E video card, at least it does for me. I have to run the PCI-E frequency higher than the default 100 MHz...