Apple to patent mixed-reality headset using RBG-D Sensors

Apple seeks to patent sensor apps for mixed-reality headset
The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. (Apple)

The US Patent & Trademark Office has published a patent application from Apple related to developing a mixed reality headset that uses advanced RGB-D sensors to produce accurate 3D images for virtual reality, augmented reality and mixed reality applications. The patent covers devices, systems, and methods that implement simultaneous localization and mapping for RGB-D sensors, such as RGB-D cameras. 

One possible implementation involves the device receiving multiple frames of a real-world scene within a field of view of a camera at multiple times, the frames including color values and depth values for pixels for the field of view of the camera. The device selects keyframes from the multiple frames of the real-world scene within the field of view of the camera. The keyframes are associated with camera poses defined in a three-dimensional (3D) coordinate system.

The device receives a current frame of the real-world scene currently within the field of view of the camera. The current frame includes current color values and current depth values for the pixels for the field of view of the camera at a current time. The device determines a current camera pose of the camera in the 3D coordinate system based on the current frame. The device then provides a virtual representation of the current frame based on the current camera pose of the camera and two or more of the keyframes.

Industry Event

Sensors Expo & Conference

Register now and save with Early Bird Rates – Limited Passes Available!

North America’s can’t-miss event dedicated to sensors, connectivity, and IoT will take place this June 22-24 in San Jose, CA. Attendees can make connections, collaborate with experts & peers, get insight into emerging technology & trends, and find new approaches to evergreen challenges. Use promo code FE100 for $100 off Conference Passes.

RELATED: Apple patent places biometric sensors in earbuds

The virtual representation is provided based on the color values and the depth values of the two or more of the keyframes.  

Apple’s use of RBG-D sensors is not entirely new. Those sensors have been used in Microsoft's Kinect, which in the early stages used PrimeSense technology now owned by Apple. The Asus Xtion Pro Live uses RBG-D sensors as well.

Suggested Articles

NPD Group says monitor sales doubled in the U.S. in the first half of March

The detection of biomarkers in feces and urine in wastewater may give a clue to the presence of coronavirus in communities infected with the virus.

Renesas Electronics Corp. has an evaluation board for developers working with the 32-bit RX23E-A microcontroller (MCU) to do IoT development.