There is no bigger showcase for new consumer-focused, sensor-driven products than the annual Consumer Electronics Show, and ahead of next week’s event haptic technology developer Ultraleap and sensor technology enabler Prophesee have announced a new system incorporating event sensors and hand tracking in augmented reality (AR) devices.
The announcement comes after Prophesee late last year unveiled its latest event-based vision sensor, the GenX320 event-based Metavision sensor. The new system leverages the GenX320 in combination with Ultraleap’s advanced computer vision and machine learning models in the form of AR glasses.
“No product currently exists in the market that leverages event sensors and hand tracking for AR,” said Ultraleap CEO and co-founder Tom Carter. “Hand interaction adds impressive value to augmented reality, not only by providing the most natural way to interact, but by expanding functionalities and making the control system more ergonomic. Low power consumption, a high dynamic range, low latency and increased privacy are all critical features to reach the glasses form factor needed for mass adoption. These are the challenges that the Ultraleap and Prophesee teams are addressing with this new product.”
The GenX320 is a 320x320 6.3-micrometer pixel BSI stacked event-based vision sensor that offers a tiny 1/5-inch optical format on a 3x4-mm die. On-chip intelligent power management modes reduce power consumption to as low as 36 microwatts, and smart wake-on-events capability means the sensor activates only when there is movement in a scene.
Prophesee has said the low-power consumption, low-latency capabilities of event-based vision sensors, which unlike frame-based sensors only capture changes in movement, are a good fit for always-on consumer products like AR glasses. The company more recently told Fierce Electronics that the Ultraleap partnership is just an initial example of what is to come as its technology is incorporated into other consumer devices, but that AR was an obvious first move.
“We will have other products we will be developing for other lines, but the core strengths of event sensors address some of the main challenges in AR today, such as the high dynamic range needed, the low power consumption, and the need for privacy,” the company said via email. “This makes them perfect for the segment.”
While Ultraleap and Prophesee are announcing the start of their collaboration at this CES, Prophesee said the companies “have been working together for a while leading up to the announcement and have published articles and presentations at events in the computer vision circle.” When Prophesee announced the GenX320 last October, it mentioned Ultraleap as an early user of the technology, but in that case the application highlighted was a gesture-tracking, touch-free ordering kiosk for a restaurant.
As for the new system for AR glasses, it can be experienced in an appointment-only product demonstration at CES that Prophesee believes will kickstart other consumer market opportunities for its technology, potentially in devices like video game consoles, TVs. smart home appliances, computers, and more.