Kuo reveals how Apple might keep real life from intruding on VR

On Friday, TF International's crystal ball wielding analyst Ming-Chi Kuo said that Apple's AR headset will use eye tracking to detect eye movements, blinks, and other physical movements that will be used to for user input replacing the need for handheld controllers. According to AppleInsider, Kuo's note to clients on Friday said, "Currently, users primarily operate the Head Mounted Display (HMD), most of which are VR devices, using handheld controllers," Kuo adds that "The biggest challenge with this type of operation is that it does not provide a smooth user experience. We believe that if the HMD uses an eye-tracking system, there will be several advantages." The eye movement data can be used to allow images and onscreen content to move in sync with a user's eyes.

With eye tracking, a user could obtain more information about an object by staring at it for a long period of time. Menus might be activated by blinking. And by tracking the position of a user's eyes, the headset's resolution could drop in areas that are not in focus thus reducing the demand for processing power where it is not needed. Kuo also said that iris recognition could be used to verify and authenticate the identities of users. The feature could also be used to complete a transaction paid for using Apple Pay.

Apple has filed for several patents dealing with eye-tracking technology. These could be used on the mixed reality device that Kuo expects Apple to unveil next year for approximately $1,000. The VR component uses 3D and other visual tricks to make it appear that the user is somewhere else and in a situation that feels and sounds realistic. He could be behind the wheel of a race car at a world-famous race track, or in the batter's box at Yankee Stadium facing a 100 MPH fastball from Marlins' prospect Sixto Sanchez. With AR, data and information is placed over a real world image. A user could follow arrows to walk from point "A" to point "B" with the real world view of the sidewalk on the screen. A box could show the time and temperature while a stock ticker could show real-time quotes on the display.

After next year's mixed reality device, we could see the Apple Glass introduced in 2025 with AR contact lenses possibly being developed for a 2030 release. Earlier this month, Kuo said that the mixed reality device will be equipped with 15 cameras along with "innovative biometrics," He also said that the mixed reality headset will employ Sony's Micro-OLED displays and optical modules to deliver a "see-through AR experience," and also provide the user with "a VR experience." The "immersive experience" on the mixed reality device will be better than those offered by rival headsets at a price equivalent to the cost of a high-priced iPhone model.

Just Thursday, Apple discussed a patent application that discusses a method that would prevent Apple Headset users from having an accidental impact with real world objects while VR puts them in a made up world. In the patent application, Apple writes, ""For example, the user may be fully immersed in VR corresponding to a pre-historic world populated with dinosaurs," says Apple. "While that virtual pre-historic world may be deficient in living room furniture, the living room in which the user is located continues to include a coffee table."

Apple's previous solution was to tell the user to remover his headset when reality is about to intrude with the virtual image. But users might not be able to do this fast enough.  "However, as experiences become increasingly immersive, the user may be unable to sense such undesirable interactions fast enough to avoid them," it says. "Moreover, abruptly removing the electronic device during an experience detracts from that experience. As such, it is desirable to address the concerns related to these undesirable interactions while also minimizing any negative impacts on the experience."

Apple's new solution calls for a picture-in-picture effect to take place. Where the "Apple Glass" wearer is in the era of the dinosaur, using picture-in-picture would allow the user to see a live view of their real life surroundings. Apple's goal is to keep the user in the AR experience until he must be alerted to his real life surroundings.