Watch a smartphone camera app take objects from the real world and paste them into Photoshop
4/10 - Cut & paste your surroundings to Photoshop— Cyril Diagne (@cyrildiagne) May 3, 2020
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ↓#ML #AR #AI #AIUX #Adobe #Photoshop pic.twitter.com/LkTBe0t0rF
The demonstration was showcased and developed by artist and programmer Cyril Diagne, who made the software in question, named "AR Cut & Paste."
In order for the software to work, a mobile app, a local server connecting it with Photoshop on a computer, and a service for detecting objects and removing backgrounds from photos are used. In addition, an algorithm for detecting where the smartphone camera is pointing at, in relation to the Photoshop project on the computer screen, helps paste the object in the correct spot.
This is an impressive look into a more streamlined future of design and image editing, and it's not a stretch to imagine Photoshop and other flagship software supporting a feature like this eventually.
Especially now that Apple added a LiDAR sensor to its iPad, its main purpose being accurately detecting objects in the world surrounding us, and supporting AR (Augmented Reality) applications, first-party support for features like this are a no-brainer, and would be a game changer.
Using Drag&Drop in #AR with @TensorFlow and #OpenCV to connect @zachlieberman prints to their video counterparts.#AIUX #WebXR #MixedReality #AugmentedReality #TensorFlowJS #MachineLearning #Interactive #openframeworks pic.twitter.com/JpZy7neDL0— Cyril Diagne (@cyrildiagne) April 11, 2020
The aforementioned AR cut and paste software by Diagne may support more than only Photoshop in the future. Unfortunately, his software showcased here is a research prototype and not a consumer user tool. And until large software companies start implementing AR like this, making it accessible to all users, we can rely on individual programmers to provide such tools for only the more tech savvy among us, if for nothing else, just for impressive proof of concept.