Not so-long-gone musician/comedian Victor Borge pointed out in a performance that his eye doctor claimed a person with one eye could see more than a person with two eyes. The doctor’s reasoning was that the person with one eye could see the other person’s two eyes, whereas the person with two eyes could only see the other person’s one eye. As absurdly logical that may sound, in the world of imaging that could be true to a certain extent.
When it comes to imaging on the road for advanced driver assistance systems (ADAS) and autonomous vehicles, the most common imaging device is a simple video camera. Of course, even non-autonomous vehicles employ several cameras that aid in backing up, parking, lane drift avoidance, as well as providing the driver with visuals of blind spots. Although cameras work fairly well for a driver behind the steering wheel, they often do not provide the whole, and very necessary picture for an ADAS or fully autonomous vehicle. Enter a second type of camera, the thermal imager.
Simply described, thermal imaging, also referred to as thermography, uses infrared (Ir) technology to create images of objects and beings from the heat emitted from them. Any object or being with a temperature above absolute zero emits infrared radiation to greater or lesser degrees, which can be detected by a thermal-imaging camera. The greater the temperature, the more Ir radiation. So, what does that mean in terms of ADAS and autonomous vehicles? Quite a bit if one is safety conscious.
In the past and most currently, thermography has been most used for military and high-end security applications. In terms of autonomous vehicles, take this into consideration. A standard video camera gives one an essentially 2D view of one’s immediate environment and does not always portray a scene in a manner that the viewer can see everything. This is not much of a problem when a vehicle is driven by a person who has eyes on the road and can focus on the most important events, i.e., forward and oncoming traffic, bicyclists, pedestrians, etc. Probably the most a driver relies on from cameras is making sure nothing is behind the vehicle when backing up or parallel parking.
An autonomous vehicle, however, is a machine that needs to be shown everything and all driving functions need to be strictly controlled via a computer system. A thermal imaging camera picks up everything, and that information can be fed into the vehicle’s control system, interpreted, and used to put the vehicle through its paces safely.
One example of a thermal imager for just such an application is FLIR’s Boson thermal camera. The Boson longwave infrared (LWIR) thermal camera cores feature a 12-µm pitch vanadium oxide (VOx) uncooled detector. Two resolutions are available, 640 x 512 or 320 x 256, lens options are plentiful.
Measuring 21 mm x 21 mm x 11 mm and weighing 7.5g without lens, specs include a pixel pitch of 12 µm, spectral range (longwave Ir) from 7.5 µm to 13.5 µm, and thermal sensitivities of <40 mK (industrial), <50 mK (professional), and <60 mK (consumer). For more specs, checkout the Boson thermal camera spec page.
For implementing the Boson in self-driving cars, FLIR offers a high-resolution Thermal Vision Automotive Development Kit (ADK) featuring the Boson thermal camera for automakers, tier-one automotive parts suppliers, and developers of advanced driver assistance systems (ADAS). Simply put, the ADK allows developers to add a long-range thermal camera to their ADAS development vehicles to advance reliability and the redundancy required for self-driving cars. As stated earlier, thermal imaging helps drivers and future self-driving vehicles see more clearly, particularly in murky environments such as darkness, sun glare, fog, smoke, and haze.
The development kit includes the Boson camera equipped with an Intel Movidius Myriad 2 Vision Processing Unit. This unit is described as a sophisticated, low-power multi-core vision processor, housed in a compact package. The kit is compatible with NVIDIA’s DRIVE autonomous vehicle computing platform. For greater illumination, checkout the ADK page. There’s also a cool little info video on that page.
Earlier today, I had the pleasure of getting an on-road demo of just what a single FLIR thermal camera can show, compared to imaging from a standard camera mounted directly next to it. Briefly put, most impressive. The thermal imaging camera picked up and displayed more useable images than the standard camera. One could easily see pedestrians, cyclists, and other relevant events clearly, in the immediate area of the vehicle and quite some distance ahead, and in dark areas and shadows, making the system highly accurate in terms of early warnings.
But hey, a picture is worth a thousand words, and a video says even more. Down below are four video examples of what I saw today in a side-by-side, thermal versus graphic imaging demo. And contrary to Victor Borge’s eye doctor, you will see more if you watch with both eyes open. ~MD