Color information is useful for recognition, but the measured image color of surfaces depends on the scene illumination. Human vision exhibits color constancy as the ability to perceive stable surface colors for a fixed object under a wide variety of illumination conditions and scene configurations. A similar ability is required if computer vision systems are to recognize objects in uncontrolled environments. We begin by reviewing the properties and limitations of the early retinex approach to color constancy. We describe in detail the families of linear model algorithms which followed. We conclude with a discussion of recent indexing methods which integrate color constancy with the recognition process.