Why We See What We Do
A probabilistic strategy based on past experience explains the remarkable difference between what we see and physical reality
Given that these otherwise puzzling aspects of the sensations elicited by the intensity of light can be understood as a consequence of a wholly probabilistic strategy of vision, we wondered whether the color sensations elicited by different light spectra might also arise in this way. After all, the distribution of spectral power in a light stimulus, which is what gives rise to sensations of color, is ambiguous for exactly the same reasons as is the overall spectral intensity. Illumination, reflectance and other factors that determine the characteristics of the light that reaches the eye are inevitably intertwined in the retinal image and cannot be disentangled.
A good starting point in thinking about color sensations in these terms is simultaneous color contrast, a phenomenon similar to the brightness contrast effects already described. Two targets with the same spectral composition placed on differently colored backgrounds serve as standard stimulus for eliciting color contrast. As in brightness contrast, the two targets look different, but now in terms of their respective color qualities: hue, saturation and color brightness. In the past, most explanations of this phenomenon relied on some sort of color averaging across the entire stimulus. As in brightness contrast, however, these schemes fail to account for the fact that color contrast stimuli can be crafted in which the same average chromatic surrounds elicit different color percepts.
An explanation of color contrast can, however, be given in empirical terms. The sources of the target and surround in standard color-contrast stimuli are profoundly uncertain, because an infinite number of combinations of reflectances and illuminants—and other less crucial factors—can generate the same distributions of spectral power. As in the case of achromatic stimuli, the visual system could resolve this dilemma by using feedback from the success or failure of the past behavioral responses to spectral stimuli. The percept elicited by a given stimulus would thus be determined by the relative frequencies of occurrence of the real-world combinations of reflectances and illuminants that gave rise to that distribution of spectral power in the past. The same argument can be applied to color constancy, which describes the related phenomenon in which the same object continues to appear similar in color despite being under different illuminants.
If perceptions of color contrast and constancy are generated in this way, then the same spectral target on two differently chromatic backgrounds would be expected to give rise to different chromatic sensations. The reason is that, in addition to requiring behaviors appropriate to the same reflectances in the same illuminant, such stimuli would in other instances have required behaviors appropriate to targets that arise from different reflectances in different illuminants. Consequently, a spectral stimulus should elicit a sensation that incorporates all possible underlying sources in proportion to their past occurrence in human experience.
To get across the merits of this way of understanding color percepts, we devised a stimulus that looks something like a Rubik's cube. By understanding the effects of spectral differences in this probabilistic way, we could generate color contrast and constancy effects that are much more dramatic than the usual textbook illustrations of these phenomena. For example, when all the information in a scene with the cube was made consistent with either yellowish illumination or bluish illumination, tiles on the surface of the cube that appear the same shade of gray in a neutral context could be made to appear either blue or yellow, respectively. This manipulation provides an impressive example of color contrast made especially dramatic by empirical manipulation of the information in the scene. Conversely, tiles that appear differently colored in a neutral setting could, by changing the probability of their possible sources, be made to look the same color, thus providing an equally dramatic demonstration of color constancy. These demonstrations show not only that color contrast and constancy are determined probabilistically, but also that these seemingly opposite effects are both manifestations of the same empirical generation of visual percepts.