Sensing technologies are changing the way that we interpret and react to the world around us. Knowing where people are and what they are doing (or trying to do) can enable sensing systems to make intelligent decisions and create environments in commercial buildings, factories and homes that are safer, more efficient and more autonomous.
One such scenario could be building manager that wants to understand the usage of a conference room or bathroom to determine maintenance schedules; another could be a manufacturer that wants to shut off machinery when a person gets too close. These kinds of sensing use cases are becoming even more prevalent in a socially distanced world, where monitoring the separation between people or identifying high-traffic areas that may need cleaning can help combat the spread of COVID-19.
Many existing sensing technologies can detect occupancy and location, including vision cameras, thermal imagers and passive infrared (PIR), but each one has performance and robustness, cost and processing complexity, or privacy challenges.
mmWave sensors deliver range, velocity and angle
Millimeter-wave (mmWave) radar sensors transmit pulses of electromagnetic energy and receive reflections back. By processing the time of flight of these energy pulses across multiple antennas, a radar system can determine the distance to targets, their relative velocity and angle of arrival. This information enables a mmWave radar to determine the position and motion of an object, making it capable of sensing people and their behavior, and also solving new detection and recognition challenges. In addition, because it uses a radio frequency (RF)-based sensing modality, a mmWave radar does not sense any personally identifiable information, which can make it a good fit for applications in sensitive environments such as homes or private settings like offices or restrooms.
New challenges in people sensing
The first problem with people sensing is motion anticipation. Although many sensing technologies today can detect presence in a room or coarsely estimate the relative location of a person fairly well, anticipating the future position of someone based on their direction and speed of movement isn’t often detected, although many applications require this information to improve safety or efficiency. For example, anticipatory requirements are showing up in applications such as elevators, where the American Society of Mechanical Engineers 17.1 Safety Code for Elevators and Escalators now requires the detection of people approaching in order to prevent someone from being struck by a closing elevator door. Now, instead of just detecting objects that may be in between the doorway, an elevator’s sensors must anticipate when an object is approaching.
Another example is an automated door at the entrance to a store. Although higher efficiency and reduced energy costs are primary goals for sustainable, energy-efficient buildings, today’s door-sensing technologies are prone to false detection, wasting energy and air conditioning with each event. With energy costs rising, architects and building managers need to pay attention to the control of doors and entryways. Opening the door only when a person’s path will take them through the door and not when they are walking perpendicular to the door can help building owners save energy and money.
Figure 1 (below) shows how an mmWave radar sensor – using range, velocity and angle-of-arrival information, along with a simple tracking algorithm – can anticipate the future position of a person and more intelligently control a door or elevator system.
along with a simple tracking algorithm, to predict the motion and future position of a person
and prevent an automated door from opening erroneously.
The second challenge is figuring out how to recognize and classify a person’s behavior. This information is valuable in applications like retail automation. Determining how customers interact with merchandise will enable retailers to more intelligently position goods in a store, or to identify areas where increased foot traffic indicates the need for additional COVID-19-based cleaning.
According to the U.S Centers for Disease Control and Prevention, nearly a quarter of all senior citizens experience a fall every year. With the majority of seniors choosing to live alone at home, many of these falls may go unnoticed for several hours and lead to hospitalization. Cameras and active infrared sensors, with the use of artificial intelligence and considerable processing power, can recognize and respond to fall events in a home, but their processing cost and invasion of privacy are extreme barriers to entry.
mmWave radar sensors use the Doppler effect to measure the velocity of objects, and when measured over time as shown in Figure 2 (below) can recognize and classify the type and behavior of objects like senior citizens without a privacy-invading camera.
mmWave radar system-on-chip (SoC) sensors, like the IWR6843 from Texas Instruments, integrate a radar transceiver front end with a programmable microcontroller, hardware accelerator and programmable digital signal processor to enable advanced signal processing and algorithms at the radar sensor edge. This integration allows for an entire radar system – including the calculation of range, velocity and angle point clouds and algorithms such as those used for motion tracking or motion recognition/classification – to exist on a single chip that sits at the sensor edge.
Rather than relying on the cloud, processing at the sensor edge enables systems to make decisions faster, with greater reliability, and with lower system cost and complexity. Some applications are simply not appropriate for cloud implementation. Imagine a horrifying scenario where the detection of a loved one failing at home depends on a reliable internet connection, or a robot arm can’t stop moving when a person approaches because it has to relay and check data with a cloud server. Because mmWave radar data can be more easily processed at the edge than other high-bandwidth data such as video, it is possible to implement processing from sensing to decision, as illustrated in Figure 3 (below) , in real time on a single chip at the edge.
the entire sensor processing chain, from RF transmission to recognition and application,
on a single device at the sensing edge.
Applications across buildings, factories, cities and homes can all benefit from sensing systems that can make more intelligent decisions based on where people are and what they are doing. Because of their unique ability to detect range, velocity and angle to a target without compromising privacy, mmWave radar sensors are opening the door to new applications in people sensing while providing flexibility and reliability with edge-based implementation.