eBay intern uses ARKit and iPhone X TrueDepth camera to develop potentially groundbreaking AR tool
by Adrian Diaconescu / Sep 11, 2018, 10:09 AM
While waiting for the necessary hardware to drive innovation forward, you’ll be happy to hear existing experimental software has been used to create a potentially revolutionary tool called HeadGaze.
This is a new type of open-source technology developed by a young eBay intern and University of California PhD Candidate, leveraging Apple ARKit resources and the iPhone X TrueDepth camera to support head motion tracking for hands-free mobile device navigation.
As you can imagine, everything’s still a little rough around the edges, but a sample of HeadGaze’s incredible potential is showcased in “example app” HeadSwipe, also available as a free GitHub download.
Basically, what reusable technology library HeadGaze does is create a “virtual stylus that follows the motion of your head (up, down, side to side)”, applying a complex method called “3D geometry mapping” to get the location of the “cursor” on the screen.
In other words, a mouse-style desktop navigation experience is enabled, with your head pointing to any location on the iPhone X screen to activate designated “buttons.” As a proof of concept, HeadSwipe lets you swipe eBay deals... with your head, and although the app looks fairly slow in a short demo video, it could mark the beginning of something big.
As lead developer Muratcan Cicek points out, HeadGaze could be used in the not-so-distant future for a number of actually convenient, non-gimmicky AR-powered tasks like scrolling through a recipe while cooking, following on-screen instructions during a messy car engine tinkering session, or simply operating your phone with normal gloves on.
All of that could be possible without getting your hands dirty, which is especially important for people with motor impairments like Cicek himself. That’s right, this guy (along with his “team”, of course) created HeadGaze and HeadSwipe without full control of his limbs. While pursuing a PhD and completing an eBay internship. What have you been doing with your life?
Posts: 1296; Member since: Mar 24, 2017
sooooo dexterity and interaction under the accessibility menu in my phone means nothing... or all those gimmicks used to interface with your phone without the use of my fingers did not exist before... interesting
posted on Sep 11, 2018, 10:16 AM 6
Posts: 1337; Member since: Oct 05, 2011
I had the same thoughts. This is not new I use these features on my Note 8 to browse the web. its cal Eye tracking. If I gaze down the page scrolls down I look either right or left to view any extra content that is not displayed totally on the screen. I mean I don't see whats new here maybe I am missing something, and I am asking an honest question, can someone explain it to me?
posted on Sep 11, 2018, 10:24 AM 8
Posts: 1296; Member since: Mar 24, 2017
Yep... and I was using this in my note 3 along with being able to wave my hand up and down to scroll through web pages and pictures... but it was a gimmick and some sites.... thought it wasn't important... We all know why it's 'groundbreaking' now don't we??
posted on Sep 11, 2018, 10:37 AM 5
Posts: 4062; Member since: Jul 23, 2013
Even with iDevices and Apple's marketing puppets pushing it, it will remain a gimmick. The issue with eye or head tracking is always error cases. You move your eyes and head too much for it to be course controls much less fine controls. It's new iPhone time so I guess it's ground breaking now.
posted on Sep 11, 2018, 7:07 PM 0
Posts: 1740; Member since: Feb 10, 2009
Marketing puppet. Really that’s a good one. What should a marketing firm do. Make there clients look bad. All you weirdo people just amaze Me with hate over an American company that’s successful. Because it’s not even founded. All there is is jealousy. You sound childish.
posted on Sep 11, 2018, 9:30 PM 0
Posts: 4146; Member since: Nov 02, 2012
Copied Samsung abilities, groundbreaking? RDF is at full strength and I imagine will be used heavily this week. It's like a war on truth.
posted on Sep 11, 2018, 10:45 AM 9
Posts: 785; Member since: Sep 30, 2016
"Grounbreaking" :)) only from the fruit company :))
posted on Sep 11, 2018, 11:37 AM 4
Posts: 1627; Member since: Jul 17, 2018
So when Samaung did na ligation, without touching the screen with features like, scrolling the beoswee window with goue wyes/head, of hovee fi gee above screen for peek feature? I recall those being blasted. But here is a copy basically and its groundbreaking?
posted on Sep 11, 2018, 6:34 PM 1
Send a warning to post author
Send a warning to Selected user.
The user has 0 warnings currently.
Next warning will result in ban!
Ban user and delete all posts
Message to PhoneArena moderator (optional):