Prophesee, a developer neuromorphic vision sensor technology, has released its fifth-generation product, the GenX320 Event-based Metavision sensor, which leverages data efficiency, a small die footprint, low power consumption to make a case to be included in a broad array of consumer IoT devices, AR/VR headsets, and other products that can benefit from motion-tracking capabilities.
Vision sensors have started to make their way into industrial machine vision applications used for security, quality control, and equipment monitoring, but in many cases a lot of power is used to capture a lot of data that ultimately is not especially useful. An example would be a frame-based security camera that collects hours of data before picking up someone entering its field of vision.
Luca Verre, co-founder and CEO of Prophesee, told Fierce Electronics, “More than 90% of the data [in such situations] is useless because there's nothing happening. Our sensor only captures what is changing in a scene, very fast, very efficiently, and independently from available light.”
Designed with functionality for integration into ultra-low-power edge AI vision devices and system-on-a-chip platforms, the GenX320 is a 320x320 6.3-micrometer pixel BSI stacked event-based vision sensor that offers a tiny 1/5-inch optical format on a 3x4-mm die. Verre said on-chip intelligent power management modes reduce power consumption to as low as 36 microwatts, and enable smart wake-on-events, meaning it will pop into action only when there is movement in a scene.
Verre believes the new sensor can help usher a new wave of consumer-facing devices equipped with low latency, touch-free interfaces operating on gesture control, motion tracking and eye tracking. These could include VR/AR headsets, but also video game consoles, TVs, computers, smart home appliances and displays, and more.
In fact, it already is being used by a range of early adopters. One of those is Zinn Labs, a company which is incorporates the GenX320 sensor into consumer projects that rely on gaze-tracking of the human eye Zinn Labs
“The new GenX320 sensor meets the demands of eye and gaze movements that change on millisecond timescales,” said Kevin Boyle, CEO and founder of Zinn Labs. “Unlike traditional video-based gaze-tracking pipelines, Zinn Labs is able to leverage the GenX320 sensor to track features of the eye with a fraction of the power and compute required for full-blown computer vision algorithms, bringing the footprint of the gaze tracking system below 20 mW.”
Meanwhile, Ultraleap has showed how the technology could be used in a gesture-tracking, touch-free ordering system in a restaurant.
Tom Carter, CEO and co-founder of Ultraleap, stated, “We have seen the benefits of Prophesee’s event-based sensors in enabling hands-free interaction via highly accurate gesture recognition and hand tracking capabilities in Ultraleap’s TouchFree applications. Their ability to operate in challenging environmental conditions, at very efficient power levels, and with low system latency enhances the overall user experience and intuitiveness of our touch free UIs. With the new Genx320 sensor, these benefits of robustness, low power consumption, latency and high dynamic range can be extended to more types of applications and devices, including battery-operated and small form factors systems, proliferating hands-free use cases for increased convenience and ease of use in interacting with all sorts of digital content.”
Prophesee also has some major partners helping it introduce its technology to key markets. Sony has manufactured previous-generation sensors for the French company, and has an ongoing co-development and marketing partnership with Prophesee–although Verre said a different, unidentified “European foundry” is making the GenX320. Prophesee also counts Intel Capital and Xiaomi as investors, and struck a deal with Qualcomm earlier this year to eventually bring its technology to smartphones.