Using High-Accuracy Color Sensors With Adaptive Displays

Sensors Insight by Dave Moon

The human eye’s discrimination of the colors of light has underscored the work of painters throughout history. Artists instinctively understand that visible light is a complex spectral phenomenon, and the proportion of red, indigo, violet, green and other color components of white light vary in sunlight. Moreover, the color of sunlight changes from place to place and from time to time. Similarly, color variations also vary widely between different types of artificial light sources, where they can be even more pronounced and have a profound effect on electronic products.

Electronic displays used in today’s smartphones, computers, and televisions, for example, can display millions of colors. Electronic device manufacturers now understand the impact of ambient light color on the perceived colors from a display and how to dynamically change these colors based on changing scene illuminants.

This article discusses the effectiveness of white-balancing technology and its dramatic impact on both the authenticity and accuracy of colors displayed on screen.

 

The effect of changes in the illuminant

The spectral content of the illuminant – ambient light – affects the human eye’s perception of the color of a viewed object. Objects viewed in daylight at noon will reveal an emphasis on blue hues. That’s because daylight is comprised of a distinct combination of both sunlight and skylight. The same objects viewed in an artificial lighting environment from an incandescent bulb (which may have a correlated color temperature [CCT] of 2700K) will appear more golden yellow. A comparison of the spectral content of various illuminants is shown in Figure 1.

Fig. 1: Spectral power distribution of various standard CIE illuminants – fluorescent light (F1) has sharp peaks at green and orange wavelengths. This is in contrast with the broad spectrum of daylight (D50, D65) and incandescent (A). (Image credit: Schwa
Fig. 1: Spectral power distribution of various standard CIE illuminants – fluorescent light (F1) has sharp peaks at green and orange wavelengths. This is in contrast with the broad spectrum of daylight (D50, D65) and incandescent (A). (Image credit: SchwartzD under Creative Commons license.)

This effect is readily perceived by the eye when viewing an image printed on paper under different lighting conditions: the colors change as the illuminant changes. However, displays do not work this way. Until the integration of ambient light sensors in smartphones and laptop computers became common practice, a display’s controller remained unaware of the characteristics of the ambient light in which the display was being viewed. For this reason, displays had a fixed pre-set white-point color temperature of 6500K of liquid crystal display (LCD) and, more recently, organic light emitting diode (OLED) screens. 6500K was represented by the CIE industry standards body as the D65 reference illuminant shown in Figure 1 and its CCT value is like bright noon-time daylight, which has a spectral power distribution with a strong blue component.

What this means is images rendered by a display’s screen will appear very similar to the same images viewed on a printed page in the same ambient conditions at noon-time sunlight. Both the display and the printed images will give emphasis to blue hues. But when viewed under warmer lighting environments, such as a warm-white 3000K LED for instance, the printed images will appear more yellow-orange, since the illuminant has a greater component of the red/yellow parts of the spectrum, and a weaker blue component.

Without an eloquent means of adjusting a display’s white point, electronics manufacturers have simply offered a single fixed D65 white point preset selection for their displays – resulting in display images rendered on the screen with the same strong blue emphasis as before.

 

Reading printed content vs. reading from a display

When reading content from printed paper, humans can read for many hours with minimal strain on our eyes. Viewing the same content from a display with a fixed D65 white point emitting a significant amount of blue light has detrimental effects – causing digital eye-strain as well as affecting the person’s ability to get a get a good night’s sleep.

Smartphone OEMs looking to differentiate in a market that recently has slowed down can now offer a new feature called “paper-like” viewing of a display. It is created by shifting the display’s white point to a warmer color temperature. Paper-like viewing is possible because of the recent availability of new, high-accuracy XYZ color sensors capable of optimal light environment color temperature measurement which is used to adjust a display’s white point from a D65 cool-blue to warmer white points.

 

The physiological effects of blue light

Paper-like viewing minimizes digital eye strain and offers the user physiological benefits during the hours before sleep by automatically adjusting the display to a warmer white point. Science has shown photobiological effects in the human eye and how blue light reception stimulates the waking-time physiology. Scientific research has shown that blue light suppresses the production of melatonin, the body’s natural relaxing agent that helps humans get a good night sleep. The absence of melatonin makes people feel awake, potentially affecting the body’s circadian rhythm cycles.

 

XYZ color science for paper-like viewing

With modern optical filter techniques, we are now able to use color filters to match the accuracy of the human eye at a cost point suitable for consumer electronics applications. Electronics manufacturers have now the capability to deploy XYZ color filters that enable paper-like technology in high volumes.  These optical filters are deposited directly onto the die of optical sensor products. Unlike traditional RGB color sensors which offer ±10% CCT accuracy in contrast, CIE XYZ color filters offer an advantage in accuracy of ±1 – 5%. 

CCT accuracy needs stem from the color space standard developed in 1931 known as the CIE xy chromaticity diagram, as shown in Figure 2.

Fig. 2: the standard CIE chromaticity diagram explained (Image credit: https://www.pinterest.com/pin/538039486704316392)
Fig. 2: the standard CIE chromaticity diagram explained (Image credit: https://www.pinterest.com/pin/538039486704316392)

Artificial light sources tend to be warmer color temperatures with residential lighting tending to be the warmest (2700 – 3100K). Office lighting is typically (3100 – 4500K).  Daylight color temperatures can range from 6000K at noon to as high as 15000K just before sunrise or just after sunset in the shade on a cloudless day. Viewing a display with a white-point that differs from its surroundings affects our perception of individual colors. A color display with a neutral or cool white point viewed in an environment with warm lighting will appear more bluish than it would appear in a cool ambient lighting environment. Adjusting the white point of the display to match the ambient lighting will minimize, if not eliminate, this effect.

The CIE chromaticity diagram captures the human perception of visible light wavelengths between 380nm to 780nm in the electromagnetic energy spectrum. Figure 3 shows the normalized spectral sensitivity of the human-eye’s cone cells of short, medium and long wavelength types.

Fig. 3: The normalized spectral sensitivity of human cone cells of short, middle and long wavelength types (Image credit: https://en.wikipedia.org/wiki/CIE_1931_color_space)
Fig. 3: The normalized spectral sensitivity of human cone cells of short, middle and long wavelength types (Image credit: https://en.wikipedia.org/wiki/CIE_1931_color_space)

This response is driven by neural responses from the short, middle and long cone cells of the retina with peak sensitivity to wavelengths in the red, green or blue portions of the visible light spectrum.  The wavelength sensitivities of the cones span a rather wide range and overlap each other; each curve is normalized in the graphic for simplicity purposes. The relative response of the three types of cone cells in the retina is sufficient to explain color vision and color can be characterized by numerous sets of color matching functions, all of which are linear transformations of the cone response functions and by extension of each other.

Figure 4 shows how the middle (M) wavelength response was then defined as a photopic view and is used to define Illuminance (in lux) because the green wavelengths are closest to what humans see. Humans are more sensitive to green and less sensitive to red and blue. 

Fig. 4: The green channel photopic response is closest to what humans see – from the CIE photopic luminosity function (Image credit: https://en.wikipedia.org/wiki/Photopic_vision)
Fig. 4: The green channel photopic response is closest to what humans see – from the CIE photopic luminosity function (Image credit: https://en.wikipedia.org/wiki/Photopic_vision)

Lux is a measure of the amount of visible light illuminating a point on a surface from all directions above the surface and is the unit of measure for brightness. The XYZ tristimulus human eye response (as illustrated in figure 5) was defined and is known as the CIE1931 2° Standard Observer and provides a connection between visible spectrum wave lengths and the physiological perceived colors for color vision. 

Fig. 5: The CIE1931 2° Standard Observer Color Matching functions or XYZ Tristimulus human eye response (Image credit: https://en.wikipedia.org/wiki/CIE_1931_color_space)
Fig. 5: The CIE1931 2° Standard Observer Color Matching functions or XYZ Tristimulus human eye response (Image credit: https://en.wikipedia.org/wiki/CIE_1931_color_space)

Color can be divided into brightness (or luminance, measured in lux) and chromaticity (measured in xy chromaticity parameters). The chromaticity diagram in figure 2 is a tool which specifies how the human eye will experience light with a given spectrum. It does not specify colors of objects since the chromaticity observed while looking at an object depends on that ambient lighting surrounding the user.

 

Human visual system

The visual system in humans is very complex and it is tightly coupled to our brains’ visual processing engine, the visual cortex. The human brain can identify the color of an object even with changes in lighting conditions. The way we see colors is not fixed; rather, it is a relative perception. When the light source type changes, humans change their “perception” of the colors viewed because there is a dynamic relationship between an object’s surface, the type of light source and our eyes.

Our visual system adjusts the relative response of the long, medium and short cone cells in response to the spectral content. Human eyes have a chromatic adaptation mechanism to understand different ambient light conditions. This is how we react to make white and grey objects look white and grey under different ambient light illuminant conditions. The optical gain adjustments for this chromatic adaptation principle is illustrated in figure 6.

Fig. 6: Chromatic adaptation
Fig. 6: Chromatic adaptation

An XYZ color sensor with its spectral power distribution (SPD) response is shown in figure 7.

Fig. 7: The XYZ Spectral Power Distribution of the TCS3430 (Image credit: https://ams.com/documents/20143/36005/TCS3430_DS000464_3-00.pdf/e7dde8f1-c089-5b48-01b8-2298637f6cfd)
Fig. 7: The XYZ Spectral Power Distribution of the TCS3430 (Image credit: https://ams.com/documents/20143/36005/TCS3430_DS000464_3-00.pdf/e7dde8f1-c089-5b48-01b8-2298637f6cfd)

The XYZ spectral response is based on the human eye, thereby providing more accurate information on how people perceive a color. While there are methods to convert RGB values to XYZ, the RGB spectral response functions are not an exact color-matching function so the resulting values from the conversion do not match the how the human eye perceives color. By closely matching the color response of the human eye, the data from an XYZ sensor can detect differences in color like the way a person would. Using a high-accuracy XYZ color sensor that outputs a measure of the CIE XYZ tristimulus values of incident light provides the best results when measuring the ambient lighting conditions. In figure 8, we show the Planckian locus within the CIE chromaticity diagram.

Fig. 8: The CIE1931 color space chromaticity diagram – illustrating the Planckian locus (Image credit: https://en.wikipedia.org/wiki/CIE_1931_color_space)
Fig. 8: The CIE1931 color space chromaticity diagram – illustrating the Planckian locus (Image credit: https://en.wikipedia.org/wiki/CIE_1931_color_space)

In the illustration, the solid curve in the middle is called the Planckian locus. Each dot on the locus corresponds to black body color temperatures with a corresponding CCT values. Adjusting the white point of the display to the ambient color temperature assumes that the display knows the color temperature of the ambient light. Since both fluorescent and LED light sources do not always fall squarely on this Planckian locus, it is better to drive the white point to the actual chromaticity coordinate values of the ambient lighting rather than defaulting to the corresponding color temperature on the Planckian locus.


Paper-Like Viewing Illustration

Figure 9 illustrates how this adaptive display technology works. In the light boxes below, two smart phones are embedded into two identical pictures. Paper-like technology is demonstrated by changing the light-source. Doing so also changes our perception of the reflected colors.

Fig. 9: Paper-like demo showing how pronounced blue light is in warmer lighting environments
Fig. 9: Paper-like demo showing how pronounced blue light is in warmer lighting environments

In the illustration, the display on the right lacks an XYZ color sensor and continuously emits D65 light. The left display has a TCS3430 color sensor accurately measuring any changes in ambient lighting conditions and a display algorithm (see figure 10) is used to enable print-like readability for display.

Fig. 10: Recommended RGB multiplier values for an environments measure color temperature (Image credit: http://www.tannerhelland.com/4435/convert-temperature-rgb-algorithm-code/)
Fig. 10: Recommended RGB multiplier values for an environments measure color temperature (Image credit: http://www.tannerhelland.com/4435/convert-temperature-rgb-algorithm-code/)

The display has an 8-bit RGB multiplier value, so the values on the y-axis range from 0 to 256 recommendations (as 28 = 256) and the values on the x-axis are the measure of color temperature values from the XYZ color sensor.  From figure 10, for a 6500K measured color temperature the recommended RGB primary display driver values should be set to 256 red, 256 green, and 256 blue – driving the display to a D65 white point. When a lower color temperature is measured from a 2700K incandescent light, for instance, 256 red, 195 green and 130 blue should be displayed.

When the 6500K light bulb is illuminated, the left display measures the ambient light, applies the algorithms recommended RGB values of 256, 256, 256 to drive the display to the exact same white point of the right display The result is both displays look the same. The printed backboard color content flows smoothly into display content for both displays.  

When the 6500K bulb is turned off and a warmer 3000K fluorescent is switched on, the ambient lighting gets warmer and the left display automatically adjust to a warmer white point to match the new 3,000K ambient light. The printed images appear more yellow-orange due to the blue light component being reduced. The perceived colors we see in the printed picture are slightly changed. The display without the color sensor continuously displays the same blue-rich D65 white point and is quite visible in the warmer 3000K environment. In this case, it’s clear how much bluer the right display looks, while the display on the left automatically adjusts its white-point conducive for the 3000K lighting environment to produce print-like readability of its display.

Turning off the 3000K bulb and turning on an even warmer 2700K incandescent results in the ambient light growing even warmer with more yellow-orange warmth due to less blue light content. Also, the left display and our perceived colors of the printed picture content are further changed. The left display automatically adjusts its white point to a white point conducive for the 2700 K ambient lighting environment in figure 11 where the right D65 white-point display emits the same copious rich blue-light content.

 

Summary

Smartphone, computer, and TV OEMs traditionally offered fixed white points for their displays and either offered a manual or time-of-day for a single preset white point – a limited effectiveness as it couldn’t cover varying lighting conditions. Fortunately, through advancements in optical filter techniques yielding human-eye level accuracy at a price point suitable to service the high-volume consumer electronics markets, a suitable means of automatically measuring ambient lighting conditions and enabling paper-like viewing of a display is now possible.

Changing ambient lighting conditions profoundly affects our perception of viewed colors in both reflected light environments and when viewing content on an electronic display. Displays with fixed D65 white points have now scientifically been shown to have physiological effects on our bodies. Automatically adjusting a displays white-point to an optimized setting under changing ambient lighting condition has proven to have noted physiological benefits – minimizing digital eye strain while helping us sleep better at night. 

 

About the author

Dave Moon is a Senior Product Marketing Manager for the Advanced Optical Solutions group at ams AG.  He has over 25 years’ experience working in the semiconductor industry and has held various applications, systems, and product definition positions at Texas Instruments, Agere Systems, Lucent Microelectronics, and AT&T Bell Labs. Dave received a Bachelor of Electrical Engineering from the University of Delaware and a Master of Science in Electrical Engineering from The Johns Hopkins University. Tel: +43 3136 500-0   Email: [email protected]