This touchless gesture control fad has to stop before it starts

This article may contain personal views and opinion from the author.
This touchless gesture control fad has to stop before it starts
Controlling our electronic devices has always been something of a challenge. There is always some sort of mythical way to interact and control our devices that seems just out of reach. We started out with physical switches and buttons, then moved onto touch-sensitive interaction with trackpads and now touchscreens. Throughout it all, there has also been work done to create voice-interaction with our electronics, which has come a long way, but is still not perfect, and can't cover all of the needs we have. Now, we have a number of different companies trying to push forward with touch-free gesture controls. 

At Mobile World Congress this past week, there was a company, Noalia, showing off another iteration of touchless gesture controls. This time, the controls were not only touchless, but didn't require the use of a camera either, as many options do. The Aramis technology shown off by Noalia detects the electrostatic energy around your hand, and translates that into on-screen gestures. This means that you can control a phone or tablet from a distance of about 10cm without touching the device.

Back in December, we saw an Israeli startup, XTR3D, that was showing off Kinect-like gesture-based controls for electronics and mobile devices, and every once in a while we've seen companies coming along and trying to make it seem as though the future is in touchless gesture control, but frankly we just can't see why. 

The case against touchless control

We can certainly understand the idea behind Kinect, which is Microsoft's gesture control system first introduced as part of the XBox 360 gaming system, and has now officially made its way to PCs as well. The logic is understandable, and it can be seen at the beginning of the video showing off XTR3D's technology. On larger screens, like TVs or desktops, where there is no inherent touch-control system and the device itself is sometimes out of reach, a gesture-based system makes some sense, although it doesn't really add any benefits over traditional controls with remotes, mice or keyboards. The trouble is that companies seem to think it is a good idea to convert this technology to mobile devices as well. Of course, once you apply touchless gesture technology to a device designed for touch interaction, you suddenly run into a number of troubles. 

First, what is the need for touchless interaction on a device that's designed to be in your hand at all times? The use case for touchless gestures on a mobile device are incredibly few and far between. Essentially, the only reasons you would ever have to use touchless gestures on your smartphone or tablets fall into just one category: touch-free means smudge-free. If your hands are dirty or if you really hate smudges, touch-free controls are a benefit, but beyond that, there is little reason to need touchless gesture controls. Touch-free screens would also make using touchscreen devices in the winter much easier when you're wearing gloves, but again, touchscreens have already evolved to the point where many can be used even through gloves (depending on thickness) without the need for special tips on the fingers. There is a possibility for using touchless gestures as a way to easily "fling" information from one screen to another, but that seems more like a gimmick used in "futuristic" technology concepts. The same thing can be done right now with either NFC, or wireless connections and traditional buttons. There is no benefit to moving that function to touchless control. 

If you're in a car, gesture controls don't help, because you're still taking your hands off the wheel, so voice-control is still the far better option. Some may say that touchless gestures will be good for touchscreen gaming, because your thumbs will finally get out of the way and allow you to see more of the screen. But, most gamers will also tell you that gaming without direct feedback isn't the same experience. It may not change what happens in the game, but being able to hit the screen a bit harder for an important movement is just flat out more gratifying than waving your hand in the air. 

Design for humans

And, that's the real issue behind this that no one bothers thinking about: hands are designed to touch and manipulate things. A wave doesn't covey the same thing as a handshake, just like flipping the bird doesn't convey the same thing as a slap in the face. We are designed to feel, touch, and interact physically, so why would we want to design products that go against that natural design? The logical extension of a touchpad is a touchscreen, but the logical extension of a touchscreen is not a touchless screen. Hands are designed to feel things and understand kinetic feedback, but touchless gestures feature none of that. A far better idea is one that we saw early in December last year, where a company called Senseg has designed a screen that can generate kinetic feedback in order to make it feel as though the screen has various different textures on it. This sort of technology has far more uses than any touch-free gesture system. 

With kinetic feedback, you can create touchscreen devices that can be used by blind people. For more casual users, games could be made where users can feel different textures. Imagine pulling back an Angry Bird in its slingshot and being able to feel the feathers of the bird, and because the technology works by creating electrostatic friction, you may even feel the tension of the slingshot band as you pull back. Similarly, if you have an app of a zen sand garden, you could actually feel the sand and stones as you move things around. Doctors could use screens with kinetic feedback to train, or musicians could practice the piano on the go and actually feel the keys on their tablet. And, there is no telling what innovations (not to mention absolutely disgusting websites) that could come to the Internet if suddenly websites could have textures and kinetic feedback.

It all comes back to the fact that we design products to be extensions of ourselves, because essentially they are just tools for us to better interact with each other, the world and possibly most importantly to interact with our own ideas. Last year, Bret Victor wrote one of the best posts on the subject that we've ever seen, and it fits into what we're driving at here, although his post focuses mostly on the fact that we only use our fingertips to interact with devices rather than our whole hands, arms, etc. The idea is still the same though: we design products in part to fit a need, but also to maximize our own natural abilities. Right now, we have designed touchscreen devices to the point that we can very easily interact with computers in a traditional way, but it is time to push forward, and we don't see that next step of evolution being to take away the touch interaction. 

Other options

More likely, the devices that are the next step in evolution for mobile are concepts like what we've seen from Nokia. Nokia has shown off two different concepts that we would love to see as a reality far more than any touch-free gesture system. The Nokia Gem was an idea for a smartphone that is 100% touch-capable, meaning we could use our entire hand for interactions. Even more impressive was the HumanForm concept which not only included full touch capability, but a flexible design for real physical interaction by bending the device. The teardrop shape may not be the best for viewing pictures or video, but obviously that's the least important thing here. 

We've also seen flexible displays multiple times, and Samsung swears that they are coming "within a year", but this only gets us part of the way there. The first couple generations of flexible screens may not see much use beyond being unbreakable touchscreens for devices that are no different from what we already have. We still need flexible components like batteries in order to be able to really build fully flexible devices. 

Either way, we need more ideas and we need to remember the core philosophy of design, which is to build things that make sense, not just things that look cool. A Kinect may make for some cool commercials, but so far it hasn't added much as far as useful interactions. Sure, it looks cool to swipe the air in order to navigate a menu, but it is in no way more efficient or faster than using a remote control. The same idea goes for touch-free gesture control. Sure, it looks cool and futuristic, but it is also completely pointless unless you are in a situation where touching the device is not an option at all. Voice control currently fits into the same category. It has come quite a long way, and can be extremely useful, but only if you're in a relatively quiet place and have an acceptable accent. 

Looking one step beyond

The best options are still out there, but are being overshadowed by these technologies which have niche uses and make for cool demos, but really don't push forward devices all that much. The transition from physical buttons to touchscreens lead to a huge leap forward in functionality because of the options available with the new method of interaction. The next step for our mobile devices will create a similar leap in functionality, because it should make just as much design sense. Touchscreens allowed for more than just pressing buttons, they opened the door to gestures and multitouch interactions. Moving to touch-free doesn't add anything, it only takes away. What we really need is to add more senses like our sense of touch with kinetic feedback. 

And that is where touch-free controls can really find a revolutionary application. Once we can combine touch-free controls with tactile/kinetic feedback, that's where we have something really out of the future. We never think about it, but all of the "futuristic" interfaces that we see in movies like Minority Report, Iron Man, or District 9, where people are interacting with holograms would be extremely difficult to use without some sort of tactile feedback. Without that feedback, you would have to be looking at what your hands are doing at all times in order to be sure you're doing what you intend. 

Remember, we saw versions of laser keyboards which would project a standard QWERTY keyboard onto a tabletop and would detect when you hit a "key". The problem was that if you can't feel the keys under your fingers, you can't touch type, which drops your efficiency a huge amount. Smartphones have been able to make touch typing without tactile feedback work because you are essentially looking at the keyboard all the time anyway. If you are interacting with a holographic object and you can't feel it, it requires too much of your attention. However, when we can combine the technology shown off by Senseg and that shown off by Noalia to create touch-free gestures with tactile feedback, that's when we'll really be seeing the future technology we all want. 



1. CivicSi89

Posts: 349; Member since: Jul 23, 2011


24. asimo

Posts: 41; Member since: Jan 05, 2012

maybe useful for disabled people somehow...

2. roscuthiii

Posts: 2383; Member since: Jul 18, 2010

You are correct, sir!

3. phones522

Posts: 94; Member since: Jan 27, 2010

i still think its cool to have as an option like what if u have a fake hand that would be useful

4. Commentator

Posts: 3723; Member since: Aug 16, 2011

I disagree. If a company puts millions of dollars into researching this kind of thing and believes there to be a market for it, all power to them. I wouldn't call for its complete cancellation. It's not like this kind of technology is corrupting society or promoting genocide or anything, and our economy is driven by new ideas. I personally don't understand the appeal of 3D movies, but some people enjoy them, and I have to accept that. To close, in the words of Gandalf: "Many that live deserve death. And some that die deserve life. Can you give it to them? Then do not be too eager to deal out death in judgement. For even the very wise cannot see all ends [for touchless gesture control]."

12. MichaelHeller

Posts: 2734; Member since: May 26, 2011

Your point with 3D is exactly what I'm saying. All of the dollars for R&D on touch-free controls, just like 3D are a gimmick aimed at nothing more than making money. There is no added value in touch-free. There can be added value in 3D (as proven with Avatar), but filmmakers have to be willing to put in the effort. Most aren't, so all we get are movies and TVs that cost more.

17. Commentator

Posts: 3723; Member since: Aug 16, 2011

I think we're disagreeing on what constitutes "value." My uncle, for instance, has an Evo 3D. He likes the 3D function that it has. Therefore, from his perspective, it adds value to the phone. These companies are assuming there are enough people out there who will find enough value in touch-free technology to buy them. Could they be wrong? Absolutely. But they could also be on the verge of something revolutionary. (And I know you wrote this article to disprove that, but forgive me for being open-minded.)

19. MichaelHeller

Posts: 2734; Member since: May 26, 2011

I hope I'm wrong and there is a use-case that I'm not imagining, but I can't see the value in touch-free gestures until there is some sort of tactile feedback to it. If I don't have to touch the screen, but I can still feel what I'm doing? That's the real revolutionary technology. As is, I can't see this as much more than a gimmick, or something that is useful in winter.

5. remixfa

Posts: 14605; Member since: Dec 19, 2008

gotta disagree. as an owner of a kinect it has very good uses... its just not useful in all situations. pretending to hold a steering wheel is pointless and counter active (oversteer and innacuracies abound) but natural hand motions and games meant for whole body movement means that gestures open up a whole new world that controllers can't touch. not sure how useful gestures on a phone would be but when you doc it to your PC or car it could be very useful

11. MichaelHeller

Posts: 2734; Member since: May 26, 2011

That's exactly the point. Kinect may have some good uses for controlling a TV on the other side of the room, but what's the value in having it on a device that's in your hand? Car doesn't help because as I said, the aim of "hands-free" in a car is to keep your hands on the wheel, which won't be the case if you're waving at your phone.

26. remixfa

Posts: 14605; Member since: Dec 19, 2008

I'm going to default to my regular argument for technology people dont see a use for. Just because we dont see a use for it today does not mean we wont have a use for it tomorrow.

6. WWThinker

Posts: 55; Member since: Feb 10, 2012

Nokia shows us two incredible concepts, what has Apple shown us? It has enough cash to buy two companies like Nokia. Yet, it seems to only have interest at milking us the consumers for profits. I would like to see Apple spending a significant portion of its cash pile on R&D, especially the long-term variants. With so much money and the faithful followings of so many fans, Apple should extend its leadership not by just making the most money than every high tech company in the world, but also tell us what the technologies of tomorrow could be, help the society (at least the USA who is in nimble) to become a better one (i.e. corporate citizenship). Steve Jobs never shows that kind of moral, would Tim Cook be a more earthly person?

7. Lucas777

Posts: 2137; Member since: Jan 06, 2011

what do u think apple does? do you think they just poop out ipads/iphones/macs without any r&d? apple is based partly on secrecy and the "surprise" and "oh that is really cool" factor… if they show us what they are going to do in the future, it kind of defeats the purpose… and i wouldnt hold your breath about apple becoming more open and "good"…

8. WWThinker

Posts: 55; Member since: Feb 10, 2012

In the history of mainkind, all the leading CE and computing companies always showed us what were in the "future" pipelines. The purpose was mostly to share concepts. Further, doing so stimulated more thoughts from the other intellects around the world, resulting in more innovations in an expedited way. If Apple keeps on the secrecy (one would argue that there is NO truly new things under the Sun especially with the flow of information over the internet), does Apple still justify the recognition of a leader? Or, it will go down the history as simply a filthy-rich merchant especially when Apple is so lacking of "good citizenship" since its inception!

29. Lucas777

Posts: 2137; Member since: Jan 06, 2011

so you are saying only truly great companies show others what they are working on… it seems like a terrible way to rate a company

13. MichaelHeller

Posts: 2734; Member since: May 26, 2011

Good job going completely off topic in order to start a fight. Very helpful.

9. paladinkar unregistered

I agree that while right now it seems a useless techonology, in a decade the use of 3D screens and gesture technology will be used as pieces to create the "futuristic" devices that we want (Tony Stark's stuff)

15. MichaelHeller

Posts: 2734; Member since: May 26, 2011

The problem is that most "futuristic" devices that people want are gimmicks in real-world use. The idea of video chatting on a phone was futuristic at one point, but how often does anyone use it? Give me futuristic AND useful. I don't want 3D screens with gesture tech. If you gave me holograms with kinetic feedback, gestures might be okay, but at that point it would be less of a gesture and more touch. The point is that even if I'm controlling something via hologram (a la Minority Report), I want to be able to feel those holograms.

18. Commentator

Posts: 3723; Member since: Aug 16, 2011

If you want "futuristic AND useful" we're going to have to start somewhere. I also disagree with the haptic feedback issue. If I were controlling holograms floating around in front of me, and they were responsive enough to my gestures, I couldn't care less about being able to feel them. I'm seeing them and manipulating them and that's enough for me.

20. MichaelHeller

Posts: 2734; Member since: May 26, 2011

That's just silly. If you can't feel what you're doing, that means you have to look at what you're doing at all times to make sure you're doing what you want to. That creates a huge reduction in efficiency. It's the same reason why we saw those laser keyboards years ago, but they never caught on, because if you can't feel what you're doing, you can't touch type, so where's the benefit?

21. Commentator

Posts: 3723; Member since: Aug 16, 2011

If it were able to translate specific hand or finger gestures into text, the haptic feedback while typing would become moot. It would obviously take a while to get used to, and may be no more efficient than handwriting something, but then again someone could devise a Swype-like system make it much faster.

22. MichaelHeller

Posts: 2734; Member since: May 26, 2011

I can see that being good for shorter messages where a gesture can be a phrase, but for longer messages, i'm still not sure. Unless of course everyone were to learn sign language. Still, I have to thank you for being my devil's advocate on this piece. I just added a bit to the end which was prompted by your thoughts.

31. bayusuputra

Posts: 963; Member since: Feb 12, 2012

for the first time i have to disagree with you, Michael.. but hey, this what makes a good discussion, right? i can't feel anything other than the cold (or sometimes hot) glass on my touch screen now, i don't have haptic feedback turned on, and i can still use my device.. in touch screen devices nowadays, we still need some visual cues as to what we are touching/dragging/tapping.. what i can see from this technology is like what paladinkar said, those 3D image projections coming out from a screen, a la Tony Stark.. i don't think we need to be able to feel those image if the response is good enough that you can twist around or bloat the 3D image with no problem, then why not? after all, in your article you said things about human senses, and the ability to see the 3D image and manipulate that with gestures seems perfectly in line with the natural human design.. yes, a handshake is better than waving, but do you also remember that when we talk, or describing something, we also use body language and visual cues to imagine things? this may not be useful for the average customers, but imagine what it can do for designers, or engineers, they can use 3D images and manipulate them to improve their designing efficiency.. we may not be able to see what this technology can bring now, none of us can predict the future, but i am sure that this technology can add benefits to our lives.. it may not be a replacement for touch screen, but it may be the perfect compliment for it, just like what voice control can compliment devices today.. it may be just a gimmick in smartphones with touch screens, but it may be the main interacting feature in devices of the future.. i can imagine a new line of samsung galaxy beam (or any other device, but now i can only think of that one) that can project 3D image coupled with this technology that can be used in corporate meetings to discuss the design of the new company product where everyone can share their ideas by interacting with the image, all by using the smartphone, which still has touch screen for its normal telephony usage.. the implementation may not be the best now, but i am sure that technological breakthrough like this can find a better applications in the future.. this is just merely the building block, to show that this technology is achievable, not just science fictions.. just like the PureView camera, people call it ridiculous, but i see pixel binning technology as something that can benefit the future cameraphones, or other cameras as well.. so this may be something in the future, just that we can't see it yet..

34. MichaelHeller

Posts: 2734; Member since: May 26, 2011

You can use your touchscreen without feedback because you're constantly looking at it. Have you ever tried doing something on your phone without looking? It's incredibly difficult.

35. bayusuputra

Posts: 963; Member since: Feb 12, 2012

the name is touch screen.. what is a screen in a device? isn't it something to look at? you were saying about natural designs and so on, but i guess sight is part of that design, and so do gestures.. we use body language, too, don't we? yes, i tried not to look at before, many occasions (especially turning off alarm), and it's hard.. maybe you are right, a haptic feedback like vibrations or even the innovative texture feedback is better, but like i said, there are other possibilities with this touchless control that we may not realise yet.. i'm not saying that your opinion is totally wrong.. i agree, at this time, it is merely a gimmick.. but try to be optimistic and think about the usage of this.. it may not be for everyone, but i believe there will be a use for this for others, maybe a special group.. maybe for the our brothers and sisters who are not as capable as we are? most of your articles really engage the both sides of the arguments, but for this one, i guess you are just bashing it.. i'm sorry to say that, although you are still one of the best tech writers out there.. so yeah, hope you understand what i am trying to convey.. :)

33. ZEUS.the.thunder.god unregistered


10. andro.

Posts: 1999; Member since: Sep 16, 2011

I have to agree a phone or tablet is a hand held device and much of the ui experience is in the tactile nature of its control. Touchless gestures is taking away the natural interaction with the product

14. Captain_Doug

Posts: 1037; Member since: Feb 10, 2012

I agree solely on the basis of precision.

16. squallz506

Posts: 1075; Member since: Oct 19, 2011

idk. im no fan of kinect or other touchless gesture systems. i do think there may be a few possibilities for the technology. #1- interaction with projections. we may not be there yet, but why not invest now? #2- 3d touch gestures on phone. imagine pinch can become "pinch and pull". #3- handheld gaming. i dont know about you michael but im sick of operating GTAIII on a touchscreen. the added dimension of gestures can give way to better controls.

23. JayRaj

Posts: 54; Member since: Oct 31, 2010

Hats off!!

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit for samples and additional information.