Is this what you're looking for?
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Class\Display
You can also edit your video card driver's .inf file with custom settings. Once you reinstall the drivers using that .inf file, the custom settings will take effect.