Google's latest AI breakthrough may turn your Pixel phone into a medical lab
According to (NIH), 73.1% of American adults are either overweight or obese. Thanks to Google's latest AI breakthrough, our next smartphone may be able to give us the necessary information to start turning that tide.
According to the National Institute of Health (NIH), 73.1% of American adults are either overweight or straight obese. That's an alarming number, but thanks to Google's latest AI breakthrough, our next smartphone may be able to give us the necessary information to start turning that tide.
Google app measures body fat using just photos
This looks complicated but once the AI has been trained, you only need to snap a few photos. | Image by Google
PhotoScan is the latest brainchild of the Google Research team, basically turning your smartphone into a medical lab using little more than its camera. The scientists trained the AI system using X-ray (a technique called dual-energy X-ray absorptiometry, or DXA) and MRI scans of actual people.
These scans can not only show the overall body fat percentage but also measure ratios between core and waist/leg fat (A/G ratio) and organ and subcutaneous fat (V/S ratio).
We demonstrate the feasibility of PhotoScan, a deep learning approach estimating body composition from smartphone photos, to predict insulin resistance with accuracy comparable to DXA scans in a clinical research setting.
- Cassie Zhou, Research Scientist, and Ahmed Metwally, Staff Research Scientist, Google Research
The researchers then combined these scans with the photos from actual smartphone cameras, showing that the model was able to visually estimate body fat with a higher degree of accuracy compared to bioelectrical impedance analysis (BIA).
Recommended For You
Basically what the researchers showed was that using PhotoScan, they were able to measure body fat using smartphone cameras more accurately than smartwatches with body composition scanners, like the Galaxy Watch family.
The system also showed very high accuracy in predicting the aforementioned A/G and V/S ratios, something a smartwatch just can't do.
Would you take unflattering photos with your phone just to measure body fat?
Predicting insulin resistance
The system was also able to predict insulin resistance with high accuracy. | Image by Google
The next step the researchers took was to see if the PhotoScan tool can predict conditions tied to body fat percentage and A/G and V/S ratios, such as insulin resistance.
The team tapped into the efforts and research already done for the Health Guardian feature on the Pixel Watch 5 and the new Insulin Resistance Trend metric.
The results were pretty encouraging and offer a non-invasive way to give users some important insights about their health.
The only problem is convincing people to snap some unflattering pictures and share them with PhotoScan. It's better than pricking your finger or going to the lab, though.
Your next Pixel phone may be a medical lab in your pocket
Your next Pixel phone may offer deeper insights into your health. | Image by Google
The research paper can be found in the scientific archive database Arxiv.org, as well as on the official blog page of Google Research. What happens next depends on how easy this system could be implemented as a commercial product, either as a standalone app or as a part of Gemini or Google Health.
Overall, our findings demonstrate the feasibility of smartphone-based body composition estimation as a scalable tool for cardiometabolic research. Clinical DXA imaging delivers the most accurate body composition but lacks scalability, whereas wearable BIA sensors offer convenience but are limited to basic body fat percentage. Our PhotoScan approach offers a promising middle ground, estimating granular body composition from standard smartphone imagery with near-DXA accuracy.
- Cassie Zhou, Research Scientist, and Ahmed Metwally, Staff Research Scientist, Google Research
But the potential is there. Your next Pixel phone may be a medical lab in your pocket, offering a convenient way to get deep insights about your health. The bigger problem is what people would do with these insights. Changing your lifestyle requires more than just installing an app and snapping a couple of pictures.
Shop Otterbox Galaxy cases!
Get Otterbox Galaxy cases, screen protectors, accessories and more. Shop now.
Mariyan, a tech enthusiast with a background in Nuclear Physics and Journalism, brings a unique perspective to PhoneArena. His childhood curiosity for gadgets evolved into a professional passion for technology, leading him to the role of Editor-in-Chief at PCWorld Bulgaria before joining PhoneArena. Mariyan's interests range from mainstream Android and iPhone debates to fringe technologies like graphene batteries and nanotechnology. Off-duty, he enjoys playing his electric guitar, practicing Japanese, and revisiting his love for video games and Haruki Murakami's works.
A discussion is a place, where people can voice their opinion, no matter if it
is positive, neutral or negative. However, when posting, one must stay true to the topic, and not just share some
random thoughts, which are not directly related to the matter.
Things that are NOT allowed:
Off-topic talk - you must stick to the subject of discussion
Offensive, hate speech - if you want to say something, say it politely
Spam/Advertisements - these posts are deleted
Multiple accounts - one person can have only one account
Impersonations and offensive nicknames - these accounts get banned
To help keep our community safe and free from spam, we apply temporary limits to newly created accounts:
New accounts created within the last 24 hours may experience restrictions on how frequently they can
post or comment.
These limits are in place as a precaution and will automatically lift.
Moderation is done by humans. We try to be as objective as possible and moderate with zero bias. If you think a
post should be moderated - please, report it.
Have a question about the rules or why you have been moderated/limited/banned? Please,
contact us.
Things that are NOT allowed:
To help keep our community safe and free from spam, we apply temporary limits to newly created accounts: