Apple to patent mixed-reality headset using RBG-D Sensors

Apple seeks to patent sensor apps for mixed-reality headset
The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. (Apple)

The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. The patent covers devices, systems, and methods that implement simultaneous localization and mapping for RGB-D sensors, such as RGB-D cameras. 

One possible implementation involves the device receiving multiple frames of a real-world scene within a field of view of a camera at multiple times, the frames including color values and depth values for pixels for the field of view of the camera. The device selects keyframes from the multiple frames of the real-world scene within the field of view of the camera. The keyframes are associated with camera poses defined in a three-dimensional (3D) coordinate system.

The device receives a current frame of the real-world scene currently within the field of view of the camera. The current frame includes current color values and current depth values for the pixels for the field of view of the camera at a current time. The device determines a current camera pose of the camera in the 3D coordinate system based on the current frame. The device then provides a virtual representation of the current frame based on the current camera pose of the camera and two or more of the keyframes.

Free Newsletter

Like this article? Subscribe to FierceSensors!

The sensors industry is constantly changing as innovation runs the market’s trends. FierceSensors subscribers rely on our suite of newsletters as their must-read source for the latest news, developments and analysis impacting their world. Register today to get sensors news and updates delivered right to your inbox.

RELATED: Apple patent places biometric sensors in earbuds

The virtual representation is provided based on the color values and the depth values of the two or more of the keyframes.  

Apple’s use of RBG-D sensors is not entirely new. Those sensors have been used in Microsoft's Kinect, which in the early stages used PrimeSense technology now owned by Apple. The Asus Xtion Pro Live uses RBG-D sensors as well.

Suggested Articles

Researchers at Johns Hopkins University’s Applied Physics Laboratory have developed a flexible lithium-ion battery designed to be non-combustible.

Raytheon Company has been selected to provide the U.S. Army with their next generation, 360-degree capable radar, called LTAMDS.

WaveSense CEO and co-founder Tarik Bolat said automakers are looking closely at ground-penetrating radar (GPR) for passenger and fleet vehicles.