Why We See What We Do
A probabilistic strategy based on past experience explains the remarkable difference between what we see and physical reality
The Basis of Brightness
The physical intensity of a light stimulus elicits sensations of relative lightness and darkness, which are arguably the most fundamental aspect of vision. Although a sensible expectation is that the sense of brightness should scale directly with the intensity of light, such that a more intense light coming to the eye always corresponds to a stronger sensation of brightness, this is not the case. In fact, two surfaces reflecting the same physical amount of light to the eyes typically look differently bright—or light—if the surfaces are observed in surrounds that are themselves returning different amounts of light. This phenomenon is called simultaneous brightness contrast.
In the past, neurobiologists based the explanation of this well-known effect on the fact that retinal neurons that send information from the eye to the visual part of the brain happen to respond more vigorously to a gray patch in a dark surround than the same gray patch on a light surround—for reasons that have to do with optimizing edge detection. If the firing rate of retinal neurons determined the apparent brightness of the patches, then the patch on a dark background would be expected to look brighter than the same patch on a lighter background.
The problem with this interpretation is that, among other things, patches embedded in scenes that have exactly the same surrounds can also be made to look differently bright. Indeed, as first shown by the 19th-century physicist Wilhelm von Bezold, a target surrounded by territory of predominantly higher luminance can—under the right circumstances—look brighter than the same target surrounded by territory of lower average luminance. This is just the opposite of the standard simultaneous-brightness-contrast effect, and the opposite of what the retinal-firing-rate explanation of brightness predicts.
How, then, can these puzzling facts about the relationship between the physical intensity of light and ensuing sensation of brightness be explained? Recall that the identical intensities of light arising from the two surface patches in question are inherently ambiguous. That is, similarly reflective surfaces under the same illuminant or differently reflective target surfaces under different amounts of illumination can generate identical stimuli at the eye. Suppose that this uncertainty is resolved entirely on the basis of past experience with what the source of such stimuli usually turned out to be, determined by the success or failure of the related behavior. Then to the extent that a stimulus of this sort is consistent with past experience of similarly reflective target surfaces under the same illuminant, the targets will tend to appear similarly bright, because things that are the same need to look the same to be behaviorally useful. However, insofar as the stimulus is consistent with the experience of differently reflective objects in different levels of illumination, the targets will tend to appear differently bright, because things that are different need to look different to be useful to the observer. Because the information in the standard stimulus for simultaneous brightness contrast is consistent with both different surfaces under different illuminants and similar surfaces under similar illuminants, then what the observer sees will reflect both possibilities. In statistical terms, the stimulus is in some degree consistent with surfaces having different reflectances, so the identical patches in the standard demonstration of simultaneous brightness contrast will look differently bright.
This may seem a strange way to generate visual percepts. Given the inevitable uncertainty of the information in the retinal image, however, this strategy may be the best—or even the only—way to resolve Berkeley's dilemma.