Apple’s upcoming mixed reality headset will feature True Depth-like 3D sensors, currently used on iPhone and iPad Pro, to recognize advanced hand gestures. The headset will feature four of these sensors, which should help it accurately capture the gestures and detect motion.
These hand gestures will be used to control the user interface in the headset. The sensors will be able to detect when the hand is covered, or in a specific direction, which will change how the user interface is controlled. Apple has been hiring engineers to work on a new application paradigm for AR and VR experiences, and advanced hand gesture detection might be one of the inputs for such apps
Apple is working on advanced hand gesture recognition using 3D sensors on its mixed reality headset
As per a new note by analyst Ming-Chi Kuo, as seen by 9to5Mac, the sensors will be able to detect people’s hands, their positioning, and even objects in front of the headset wearer.
We predict that the structured light of the AR/MR headset can detect not only the position change of the user or other people’s hand and object in front of the user’s eyes but also the dynamic detail change of the hand (just like the iPhone’s Face ID/structured light/Animoji can detect user’s dynamic expression change). Capturing the details of hand movement can provide a more intuitive and vivid human-machine UI (for example, detecting the user’s hand from a clenched fist to open and the balloon [image] in hand flying away).
To accomplish such advanced detection, the sensors will have an increased field of view, compared to current Face ID TrueDepth sensors, which will allow them to detect objects and gestures up to 200% far away. It is important to note that Kuo had previously said that Apple’s mixed reality headset will feature 15 camera modules.
Along with an innovative hand-gesture detection system, Apple’s mixed reality headset is also expected to have alternative inputs in the form of eye tracking, iris recognition, voice control, skin detection, facial expression detection, and spatial detection.
Apple’s first-generation mixed-reality headset is expected to launch in 2022, and will feature a chip with Mac-like performance. The mixed-reality headset’s features will be focused on gaming, communication, and media consumption.
Read more:
- Apple’s AR/VR headset will be supported by Wi-Fi 6E connectivity for faster speed and low latency
- Apple’s premium AR/VR headset to launch in 2022 with a hefty price tag
- Apple’s mixed reality AR/VR headset might go in production in Q4, 2022
- Apple explores high resolution micro OLED display with 3,000 PPI for its VR headset
