Measuring realism perception of generated imagery?

Question

We are working on a complex system, which will take data from a game, process it via machine learning methods (think types of neural networks (NN)) and output an enhanced image.

The enhancement varies quite a bit depending on the scenario - super resolution, image-to-image translation (e.g. removing objects from the background or completely converting an image from one domain to another (think DALL-E)) and so on. All NNs can be measured with some sort of metric (PSNR, SSIM, FID and so on). However those do not measure the realism perception of the viewer of the final imagery. In fact there are many cases, where metrics show top scores yet the viewers would dismiss the imagery and label it as "unrealistic" withing milliseconds.

One example for an attempt to "add realism" can be seen here - the Photorealism Enhancement project led by Intel. While the result looks more visually appealing (for me at least) I am looking for an actual quantifiable way to measure how the viewer perceives the result.

I looked into an old paper called Measuring the Perception of Visual Realism in Images where they introduce the so called Realism Response Rating. I am hoping that someone here can point me to more current research or examples.

PS: I know that realism is sort of subjective hence cannot be 100% accurately measured. :P

Answer 1

The only thing I am aware of when it comes to the understanding of 'realism' is the concept of 'uncanny valley' that is used to describe the different degrees of emotional response to a stimuli that relates to human-likeness.

But I think you can apply a similar approach where you can chart a range of emotional responses and try to see if there is some pattern or grouping that describes the general range and degree of responses that you capture.