Apple to patent mixed-reality headset using RBG-D Sensors

Apple seeks to patent sensor apps for mixed-reality headset
The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. (Apple)

The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. The patent covers devices, systems, and methods that implement simultaneous localization and mapping for RGB-D sensors, such as RGB-D cameras. 

One possible implementation involves the device receiving multiple frames of a real-world scene within a field of view of a camera at multiple times, the frames including color values and depth values for pixels for the field of view of the camera. The device selects keyframes from the multiple frames of the real-world scene within the field of view of the camera. The keyframes are associated with camera poses defined in a three-dimensional (3D) coordinate system.

The device receives a current frame of the real-world scene currently within the field of view of the camera. The current frame includes current color values and current depth values for the pixels for the field of view of the camera at a current time. The device determines a current camera pose of the camera in the 3D coordinate system based on the current frame. The device then provides a virtual representation of the current frame based on the current camera pose of the camera and two or more of the keyframes.

Sponsored by Infosys

In Conversation with Antonio Neri, President & CEO – Hewlett Packard Enterprise & Salil Parekh, CEO – Infosys

Hear the CEOs of Infosys & HPE discuss the current crisis and how it has accelerated the need for digital transformation for their clients. Connectivity at the Edge, right mix of hybrid cloud, ability to extract data faster than ever before… these are just some of the contributions that HPE and Infosys make to our clients’ digital transformation journey.

RELATED: Apple patent places biometric sensors in earbuds

The virtual representation is provided based on the color values and the depth values of the two or more of the keyframes.  

Apple’s use of RBG-D sensors is not entirely new. Those sensors have been used in Microsoft's Kinect, which in the early stages used PrimeSense technology now owned by Apple. The Asus Xtion Pro Live uses RBG-D sensors as well.

Suggested Articles

Revenues overall hit $3.82 billion, up 1% from third quarter of 2019, as auto plants reopened and personal electronics revenues grew

MIT Sloan and Boston Consulting Group call for expanding organizational learning to gain better financial rewards of AI deployments

Originally a 1960s memory manufacturer, Intel wants out of NAND following the market decline in 2018