Introduction
The widespread application of sensors has enhanced our electronics experience for well over a decade. Sensors are now central to applications in aircraft, automobiles, smartphones and consumer electronics. They are transforming the performance of engines, machines, industrial robots, mobile phones and personal connectivity devices into smart systems.
Fig. 1: Application Specific Sensor Nodes (ASSNs) simplify and enhance designs utilizing the integration of data from multiple sensors.
Sensors have now proliferated to the degree where the development of highly intelligent devices that have the potential to change how we live is now possible. However, it is necessary to address this complexity of sensor interaction and management in a system to optimize performance and reliability and achieve full potential.
It is clear that with an array of distributed sensors, each having individual operating parameters and support requirements, development and verification time increases proportionally to the increased design complexity. This complexity moves beyond hardware interaction considerations into software where efficient co-design strategies need to be employed to process sensor data into the overall intelligent system output. This article discusses the main points for consideration in designing next generation sensor-based systems, and emerging techniques for optimizing system performance.
Analysis
In any system, each installed sensor performs a specific and well-defined function. Sensor fusion algorithms process data streams from each independent sensor and combines them to provide higher-level and more accurate information. For example, output data from accelerometers, gyroscopes, magnetometers, and even pressure sensors can all be combined together or 'fused' to provide a single location or orientation result. While each individual sensor is carefully selected for a particular application, it is the effectiveness of the sensor fusion algorithm that delivers the simple and precise information needed to implement the application.
Fig. 2: Sensor-fusion software intelligently fuses raw data from multiple sensors for optimal performance.
Many applications now require an array of sensors to provide clear and unambiguous information to measure position, movement, rotation, temperature, pressure, etc., and the sophistication of these applications is increasing. These can be found in industrial applications such as robotics and logistics, hobby market sub-systems, or in the vast consumer market in our everyday smart phones and wearable devices.
Solutions
OEMs are constantly striving in their quest to create technically superior and differentiated features in order to gain traction in this fiercely competitive market. Mobile system designers are focusing their efforts on creating always-on systems, and the newest generation of sensors will be vital in making this a reality. Unfortunately, the increase in performance and functionality comes with a penalty, which is without optimization, battery life is reduced significantly in proportion to processing complexity, and this is not tolerable for the end customer. The challenge for the designer is to be able to deliver the added feature-set without compromising battery life. It is vital that the sensor fusion operation consumes as little power as possible while maintaining responsiveness; a consideration that must be addressed through the hardware and sensor fusion design process.
For the designer, it can be a daunting task when considering the fusing of sensors in a new system. There are numerous software companies providing sensor fusion software, some providers claim that their solutions are "sensor-agnostic." In reality, since every sensor is unique, with its individual characteristics concerning response, noise sensitivity, temperature dependency and other critical properties, an effective algorithm takes those individual characteristics into account and can be fine tuned to the specific hardware component. Naturally, the most accurate sensor fusion comes from an intimate knowledge of the sensors being fused – knowledge that normally only the sensor manufacturer will have.
Making Smartphones Truly Smart
With Smartphone architectures and configurations becoming increasingly standardized, it is harder for manufacturers to achieve vital competitive differentiation. Sensors are so central to what makes a phone smart, they have become a powerful strategic enabler to the smartphone manufacturers. Consequently, the accompanying sensor fusion becomes of strategic importance too. Smartphone makers see great potential here and want to utilize these advanced algorithms to differentiate their phones. Of course, there will be trade-offs in terms of functionality, accuracy and battery life, but to allow customization and therefore differentiation, sensor fusion IP must come equipped with variable parameters that can be adjusted by the smartphone makers who can make their own tradeoffs to create phones that will out-perform competitors in a crowded market.
Going back to our earlier concerns regarding power and battery life, the gyroscope poses a particular challenge because it has a mechanical element that must be constantly driven and therefore consumes power in order to measure turns, and is a relatively power-hungry component. As a result, a great deal of attention goes into the development of the sensor fusion algorithm to find ways to reduce gyroscope power consumption.
One common method is to put the gyroscope into sleep mode during periods of inactivity but allow the low-power-consumption accelerometer to run continuously. When the accelerometer detects motion, it wakes up the gyroscope. The accelerometer must be able to awaken the gyroscope quickly enough to track motion which requires a fusion algorithm that is tightly coupled to the specific timing of the sensors.
To reduce power and cost used in many mid-range and entry-level smartphones, another approach is to eliminate the gyroscope entirely using a fused accelerometer-magnetometer combination as a "soft gyro." This takes particularly advanced fusion techniques to provide adequate responsiveness and immunity to local magnetic "anomalies" due to close proximity of interfering metals or magnetic fields that would degrade the accuracy of the reading. Just note, these are anomalies that the gyroscope would otherwise have compensated.
Fig. 3: Simplified implementation of Smart Fusion techniques.
These examples, while simplified, give us an insight into the need for fusion. In practice it is more complex and a real challenge for the designer.
In a "real world" design there are a number of sensor combinations that may need to be fused: six axes (three-axis accelerometer + three-axis gyro or three-axis accelerometer + three-axis magnetometer), nine axes (three-axis accelerometer + three-axis gyro + three-axis magnetometer), or ten axes (with addition of a pressure sensor). Other measurement applications include humidity, temperature, and a MEMS based microphone. With this diverse application set, it is vital that system designers have access to sensor fusion software that is able to accommodate all these configuration disciplines in a straightforward, time and cost efficient way for their specific projects.
The smartphone is probably the most sensor dependent consumer device ever developed. It originally relied on one main processor, the application processor (AP), for all general-purpose computing including sensor management and fusion. The AP is a notoriously power-hungry part of the system that has led to the evolution within the smartphone architecture to include low power sensor hubs that can be kept awake to manage sensor functions and fusion while the AP sleeps. This process of running the low power sensor hubs while the AP sleeps enables the development of an intelligent consumer device without paying the high and unacceptable penalty of reduced battery life.
Wider Applications In The Consumer World
Aside from our smartphone example, sensors are rapidly being deployed into a diverse range of other consumer gadgets, from fitness trackers and health monitors which can be customized for wearable computing applications, through to special-purpose navigation systems. Unlike the smartphone, a pre-standardized architecture does not exist. Whereas smartphone makers invest time and personnel in strategic sensor technology and long-term licensing agreements with sensor and software vendors, these other consumer product manufacturers have much shorter-term goals. They have to innovate at a rapid pace, with tight market windows and short product lifetimes. They simply do not have the time, funding or inclination to become sensor experts. They may change sensor strategy often as they move from one product to another.
System designers require a different approach to sensor fusion. They may want to make some high-level tradeoffs, but the fusion algorithms will not be their prime differentiator. Their systems will stand out based on technology built on top of the basic sensor fusion. Essentially, they want a solution that just works.
One approach for this is through what Bosch Sensortec calls an Application-Specific Sensor Node (ASSN). This is a new category of device that acts as a complete sensor hub subsystem, with sensors and sensor fusion algorithms built-in. The company's BNO055 device is already available for orientation, and, to a system integrator, it looks like a purpose-built orientation unit.
The details of the sensor fusion routines become a non-issue since everything including software is buried inside. Another exciting device from, the BN0070, is now available and has been specifically developed for the growing head-mounted-displays (HMD) market.
Fig 4: ASSNs prove to be the fastest route to market for the developer.
Having sensors and sensor fusion housed in the same package ensures that these elements have been optimized to work together. A system designer does not need to invest valuable time assembling, optimizing and debugging such a subsystem because by design, everything is optimized for the highest accuracy and lowest power consumption.
Since ASSNs use internal software to run the algorithms, they could also be a convenient place to run other low-level code completely isolated from the sensor fusion portion. Another advantage for the designer is that sensor fusion code is isolated and protected within the ASSN. This eliminates the need for expensive software licensing deals that often require months of work to pass through legal departments and result in high costs and delays to product launch. An engineer can simply acquire the ASSN, implement the system, and go into production.
Conclusion
Sensor fusion technology is now reaching mainstream status even as steady innovation continues. Bosch Sensortec has contributed greatly to the development of a flexible and relatively easy process to provide stable sensor fusion IP and can address long-term strategic needs of smartphone manufacturers and accommodate the short-term tactical requirements of a diverse set OEMs.
When sensor fusion algorithms are delivered as licensed software and as off-the-shelf ASSNs, it means that sensor fusion can more easily become adopted as the preferred system solution for manufacturers involved in the increasingly wide range of consumer devices that rely on sensors for their successful application and market differentiation.
Related Articles
Location-Based Sensor Fusion on the Rise
Camgian Releases M2M Sensor Fusion Engine
Sensor Platforms Adds Pedestrian Dead Reckoning to Its FreeMotion Library
Vivante Unveils Area-Optimized GPU
World's lowest power microcontrollers now deliver big benefits in tiny packages
About the Author
Keith Nicholson is Senior Manager in the Strategic Marketing Department of Bosch Sensortec GmbH. He studied Electrical Engineering and Computer Science at Edinburgh University (UK) and has a Master in Business Administration from Vlerick Management School in Leuven (Belgium). Before joining Bosch Sensortec, he has held various engineering and product marketing positions within the semi-conductor industry, most recently as a senior product marketing manager for automotive micro-controllers at Atmel.