I guess one would have to know the actual mapping function used, or test the panel / Card combo.
Normally scaling is only done by one thing, with the Nvidia control panel you can set specifically if the GPU will scale for you, which means the monitor receives a signal in its native res and does no scaling of its own.
OK so I've done my first test, running the test images I mentioned before by simply putting it as the desktop background has indeed revealed bad scaling, I set the native 4k wallpaper and got an obvious black/white divide in the pattern. However taking the 1080p version and setting the wallpaper to fit, windows scales it bit the whites become grey.
I took screenshots of the desktop in both cases and this is the result:
This occurs with both Nvidia panel set to both monitor and GPU scaling and it makes no difference, so I suspect that windows scales the image internally itself, badly.
Question then becomes does windows use the same bad scaling for everything or is the desktop wallpaper one unique. I'm not sure.
I realised that VLC player allows you to display images full screen so I did the same test loading both images and again the same thing, the 2160 is noticeable white but the scale 1080p is grey both on the blacks and white. How does VLC do its scaling I wonder?
The VLC settings have the accelerated video output (overlay) setting ticked which is basically using the hardware acceleration of your video card. The Output setting for me was automatic. I've tried setting it to D3D specifically and OpenGL specifically but get the same results.
Looks like scaling by windows is pretty naff.
So I set my screen res in windows to 1920x1080 and I set my Nvidia control panel to scale by aspect ratio (so I get full screen) and tried both GPU and display scaling, both give me clear black and white using the 1080p background, so both the Nvidia drivers have no problem outputting the native res to be 1080p and scaling the output to the monitor.
perceivably when looking at the monitor either way you see grey, its just too fine of a resolution to discern the pattern, but you can see it's a much lighter grey when using the native pattern as your background rather than the scaled one so it's immediately obvious the difference.