This article may contain personal views and opinion from the author.
What piqued our interest in the phone was the enigmatic "better spatial perception" when explaining what the four rear cameras will bring to the 5G S10. Samsung, it turns out, has been developing time-of-flight (ToF) camera sensors
to go into its smartphones this year. The tech is different than the 3D structured light face-reading kits on the front of phones like the iPhone XS
or Mate 20 Pro
in that it can map a whole room full of objects for you. It's different than popular mobile AR efforts like Pokemon Go, too.
ToF cameras are the next step in the mobile AR game
There are already phones on the market with a ToF rear sensor, the Oppo R17 Pro
and Vivo NEX Dual Display
edition. The TrueDepth
camera setup on the iPhones uses 30,000 points to develop your 3D face matrix, while the ToF cameras on the Oppo and Vivo use a sensor and auxilliary light kit that can detect and map up to 300,000 points at a distance of about ten feet, bringing about more opportunities in 3D space sensing.
The smallest Time-of-Flight sensor from Infineon/PMD is already available in phones and has better performance, smaller size, and gentler power draw than the TrueDepth set, so it's not hard to imagine Apple using this technology for a rear 3D-sensing camera, although it is said to go with Sony
for the actual sensor.
The interesting part is that the Germans from PMD who are providing the ToF camera tech for Vivo and Oppo just recently boasted with opening new offices in China and Seoul. Does that mean we will see the S10 5G use the same ToF camera supplier?
Samsung's ToF camera sensor may debut with the most souped-up Galaxy S10 model in a mid-February Unpacked event, but it will actually trickle down to the midrange A-series, too, tip insiders. Apple is also rumored to equip the 2019 iPhones
with a 3D-sensing rear sensor for AR/VR purposes, so the time-of-flight camera fights may officially begin this year.
What could a 3D-sensing ToF camera do? In perspective, it can help with gesture navigation and create augmented reality (AR) experiences, like trying on new clothes in the virtual domain, or even in placing virtual IKEA furniture around the house in real time. We played around with the one on the back of the new Vivo NEX Dual Display
edition to check the currently available features that one brings.
Spacial perception is your digital ruler
AR sticker aided by Vivo's ToF camera
So far, besides some souped-up AR stickers, the easiest way to demonstrate the ToF camera virtues on the Vivo is to fire up its AR Ruler app. It displays the distance between objects, but so do Androids or the iPhone measuring app that use the regular rear camera, that is why we pitted the ToF distance against an iPhone and a good ol' analog ruler. The results? Well, the Vivo will indeed measure the distance between objects that are further away, and you don't have to move with the handset like with the iPhone if you want your starting point to stay where you pinned it.
When it comes to the actual results, on closer distances they are pretty spot-on, about the same as a physical ruler. Move further away, though, and the iPhone gets way off or won't let you take a measurement at all, while the Vivo can go across the room and tell you the distance between objects. The ToF camera's final results, however, were often conservative compared to our analog ruler, so, while its depth crawl gives an advantage, it's still not as precise an instrument as one might desire.
ToF depth-sensing camera vs regular camera vs ruler measurements
In theory, ToF sensors will allow you to change digital clothes, play augmented reality games, and gesture like a madman while you serve Kinect-style tennis backhands against your roomies.
In reality, the current ToF cameras on phones from Vivo or Oppo need more specialized software and apps that will take advantage of them, which will hopefully be spearheaded when companies like Samsung and Apple include time-of-flight sensors in their main camera sets.