Apple and iOS 11 could revolutionize smartphone photography with a next-generation image file format


At this year's WWDC, Apple announced that it's adopting two new standards-based formats for storing and processing images and video, called HEIF (High Efficiency Image File Format) and HEVC (High Efficiency Video Coding) respectively. Although the implications of both for the future iOS cannot be understated and merit their own research, today we will be focusing on the HEIF standard, which by the way is pronounced "Heef," according to Apple, because we all know how obsessed people can be over the pronunciations of different file extensions (*cough* GIF *cough*). The finnish engineers who came up with it would likely beg to differ, but anyway.

HEIF: what is it, what does it do, how to use it, and should you care in the slightest?


Without going into too much technical mumbo-jumbo, JPEG is still the de facto standard for image compression and is universally supported. However, several new compression algorithms have cropped up in recent years, that are capable of shrinking file sizes much more efficiently than JPEG, without sacrificing the perceived quality of the picture. HEIF is one such format and it's capable of compressing images twice as efficiently as the ancient (in tech years) JPEG standard, which results in images that retain the same visual quality but are on average two times smaller. Not up to two times smaller, but on average two times smaller, Apple claims, which is a somewhat important distinction, especially considering how the visual quality of HEIF images is supposed to at least as good as JPEG. But there's more to HEIF than that, as it also supports stacks of images—and by extension, animations—as well higher bit depths than JPEG (16-bit color), embedded HDR metadata, depth maps, and many other bells and whistles that.


So, where's the catch? There has to be one, right? Well, yes and no. HEIF is a promising new format with many advantages over JPEG, but as of yet it is not universally supported, meaning that you can take true advantage of it on a limited number of devices only. What we mean is that, say, if you want to connect your iPhone to your Windows PC and just transfer the photos over, they'll be converted to JPEGs, losing much of the initial benefits of using HEIF in the process.

The future of HEIF is bright, as long as it gets widely adopted—and it likely will, because Apple—but let's focus on the now. What happens if you buy an iPhone 8 or 8 Plus, or an iPhone X at launch, and want to view your photos on non-Apple devices? What benefits are there to using the "High Efficiency" setting over "Most Compatible," and should you even care?

"High Efficiency" vs "Most Compatible"


These are the two new camera settings available in iOS, that you can find under Settings > Camera > Formats. "High efficiency" is the current de facto setting when you update to iOS 11, which captures your photos and videos in the HEIF/HEVC formats, while "Most compatible" saves in good ol' JPEG and H.264. The HEIF new format will obviously leave you with smaller file sizes on your iPhone, but what about image quality? Let's dive right in!

For the purpose of this article, we took some 70 photos of different scenes using the two compression methods and, and upon transferring the images to a computer, we found that images captured using the "High efficiency" setting were on average 20% larger in file size than their JPEG counterparts. Wait, what? We thought HEIF was all about compressing images more efficiently, yet photos captured in this format are larger? Again, yes and no. When you transfer HEIF images to a device that does not support the format, such as your PC, the images are actually converted to JPEGs and thus increase in file size, so as to preserve their higher quality, while the HEIF photos remain smaller on your iPhone. In fact, they indeed take up, on average, 50% less space on your device, just like Apple said they would. So, the photos you're seeing here are conversions of .HEIC files, but the originals are avai

We began in a controlled environment, by snapping two identical shots of our beloved ISO 122233 chart with the iPhone 7 Plus, from a tripod. And, well, they do look pretty much identical when viewed in full size, but let's see what happens when we go deeper... like, 300% deeper.



To download the original JPEG and HEIC files for this scene, follow this Google Drive link.

Huh, well, from this close up, both look equally crappy in their own way, but the JPEG is 2.5MB, while the HEIF photo weighs in at just 1.2MB. So, at full size, we are unable to detect any meaningful differences between these two photos, but one is nearly half the file size of the other. That's pretty neat! But let's move on to different, more dynamic scenes.

Next up, two shots of a cathedral at mid-day:



To download the original JPEG and HEIC files for this scene, follow this Google Drive link.

No discernable difference here at full size either, as far as detail is concerned, although the JPEG has is 3.4MB, while the HEIF is only 1.8MB. We're going to have to take a closer look again, so as to see the differences between the two compression methods on more a granular level. As for why this JPEG file is larger than one from the previous scene, that comes down to detail. The ISO chart is a primarily white sheet of paper with some shapes printed on it, while the cathedral's facade packs not only more detail, but features numerous colors as well. Keep in mind that evenly colored areas, such as the blue sky in this scene, or white areas on the ISO chart, lend themselves quite well to compression and can thus result in vastly reduced file sizes. A photo of a white wall won't be the same size as a photo of a face, for example.



To download the original JPEG and HEIC files for this scene, follow this Google Drive link.

In this scenario, the file size discrepancy is again quite pronounced, as the JPEG weighs in at 2MB, while the HEIF file is just 1.3MB large. As far as image quality is concerned, color- and details-wise, we were not able to detect any meaningful differences while examining the photos (all of them, not just these) on various devices and displays, including iPhones, iPads, Macs, and PCs.

Bottom line


Similarly to how Google Photos lets you save all the images you want to the cloud at the cost of a slight hit on quality, the new formats are currently useful only if you want to save space on your iPhone and iPad, though they have the upper hand, as quality doesn't suffer. The good news is that, no matter whether you're sticking with Apple's Photos app and iCloud, or would rather go with Google Photos for your cloud backups, HEIF files will retain their original size and can be viewed on compatible devices (or the web) on both platforms. What's not so nice is when, say, you want to transfer some photos over to your computer and you and up with bigger files that carry over very few of the benefits of the originals. Or, perhaps even worse, you open Google Photos on your PC and download the original HEIF files that you can't view or edit, or do anything else with them really.

As it stands, the main advantages of HEIF/HEVC over JPG for Apple users are currently storage-based, but this will change in the future, as the format becomes more widely adopted. For example, HEIF supports auxiliary images, such as alpha, disparity, and depth maps, meaning that images preserve depth data, which could potentially allow users to apply edits over different "planes" of an image (e.g. foreground, middleground, background) on any compatible device post-factum. Yes, you'll be able to apply your favorite "bokeh" effect to any photo! But further than that, HEIF also supports a wider color gamut, meaning that, provided you are viewing the photo on a display that's capable of covering the wider gamut, photos will have richer colors, scenes will look more dynamic, and less banding will be visible in smooth gradients. But that's just a handful of examples. HEIF also supports multiple images in the same file (for example, stereo 3D images or multiple exposures), as well as multi-resolution representations of the same image for faster previews and conserving space, animations, and the ultimate goal is to have it support even different media types, such as audio and text.

If you haven't seen a 2.9 gigappixel (yes, you read that right, GIGAPIXEL) panorama with a resolution of about 91,000 x 32,000 pixels and a TIFF file size of over 2GB, crunched down to 160MB in HEIF and easily viewable on an iPhone 7, then you should absolutely watch "Working with HEIF and HEVC" session from WWDC 2017.

The real benefits of HEIF will shine when computational photography becomes commonplace—or at least more so than now—and the format becomes more widely adopted, but this is still quite a way off. For now, and as far as us regular users are concerned, the new format will indeed save storage, but it will also cause headaches for many due to compatibility issues.

FEATURED VIDEO

71 Comments

1. cdgoin

Posts: 614; Member since: Jul 28, 2010

Right.. This will be the new standard because.. Apple..? Are you on crack ? AAC is a universally used format WHERE ? Other than Itunes ? Thats right NO WHERE.. Name ONE time Apple created a universal standard ? I can't think of any. They steal others ideas & repackage OR create a new idea and then lock it down so no one wants to use it. Heck the New X other than a proprietary chipset, is lower than most Galaxy handsets and Sony ones. Only way someone KNOWS the Apple Chipset will be faster is due to running tests and taking Apples word on it. Real world I doubt its much of a difference.

2. Rampage_Taco

Posts: 1049; Member since: Jan 17, 2017

I was thinking the same thing, AAC was just apples way of trying to lock their audio files within iTunes and the Apple ecosystem. Can't help but think that's their idea here as well, Check out our great new file type... oh but if you want to share it with friends, they need to have iPhones too...

49. Finalflash

Posts: 4063; Member since: Jul 23, 2013

They are going to start their decline now and it will largely start with squeezing their customers as hard as they can until people finally get tired of being stupid and quit Apple products.

51. sgodsell

Posts: 7235; Member since: Mar 16, 2013

If you actually look at video side of HEVC it has roughly the same compression bit rate as VP9. The only difference is VP9 is open, where as HEVC OEMs have to pay a fee per device for its usage. The only browser that doesn't support VP9 today is Apple's Safari. Well Microsoft's older Internet explorer didn't either, but Edge supports VP9. So HEVC is not going to be a standard. I can guarantee that one.

5. PhoneInQuestion

Posts: 496; Member since: Aug 20, 2017

"It's okay if Apple sticks to proprietary nonsense but if anyone else does it it's stealing!!!"

27. Nopers unregistered

This isn't even proprietary it was developed by MPEG.... The only reason most other people use JPEG is because of licensing fees.

19. mikehunta727 unregistered

"Only way someone KNOWS the Apple Chipset will be faster is due to running tests and taking Apples word on it. Real world I doubt its much of a difference." A test like editing a 4K video in Lapse It will show the massive difference in performance granted if it's capable of using more than 2 cores on iOS(has iPad support so I believe it does), we see the A10 processing 4K videos in Lapse It minutes faster than any other smartphone. But yeah otherwise, you won't be able to tell much of a big difference in normal usage, it's only in heavily multi threaded scenarios where you will really feel the difference(like example above with Lapse It), and well newer games that leverage it/AR applications/whatnot

30. steodoreben

Posts: 379; Member since: Sep 26, 2013

This PA article is misleading. PA wanted people to believe that Apple invented HEIF and HEVC. Apple's new OS supports HEIF and HEVC.

32. Milen_Y

Posts: 112; Member since: Jun 09, 2016

Currently on pure H2O, thanks for asking. Anyway, Apple is simply pushing for the adoption of the two formats by adding support for them in iOS 11 and macOS High Sierra. A format like HEIF is needed in this day and age, as JPEG is simply not fit for the purposes of computational photography. Whether the format will see a wider adoption remains to be seen.

34. hemedans

Posts: 755; Member since: Jun 01, 2013

And you should mention Nokia Co develop HEVC and HIEF is based on HEVC. So instead of praising Apple here we should thank company like Nokia for bring technology like this

47. Xilam unregistered

Are you on crack. You should instead be angry at AEMs, when are they gonna push new technology standards instead of clinging to file formats that take up twice the space. If that was Apple was sitting on its ass like android (not adopting the new standard to free up space on our phones) all you guys willl be chanting that Apple is pushing you to buy higher capacity phones, greed, and all that garbage. But because android is trailing behind - "Apple is proprietary" and s**t. For crying out loud, keep you effing wires and jacks, keep your SD cards, keep your buttons, keyboards,and CD drives. But don't redicule those that prefer to moving forward.

54. BlackhawkFlys

Posts: 908; Member since: May 07, 2014

If you look at the sample images above it is obvious that the HEIF looses lots of details

62. razmahtaz001

Posts: 501; Member since: May 11, 2013

re: 54. BlackhawkFlys (Posts: 649; Member since: 07 May 2014) If you look at the sample images above it is obvious that the HEIF looses lots of details ================================== thats what i notice too...the jpg pics seem to be slightly brighter at minimal and contrast seems slightly higher as well...atleast to my eyes it does LOL

57. srgonu

Posts: 543; Member since: Feb 13, 2012

With not following universal standards and trying to be monopolist, apple cannot and will not go far. Remember this article and check a year later to find out this is an erroneous and iSheeple prediction.

67. bmwzimmer

Posts: 7; Member since: Sep 18, 2017

You don't think the new A11 chipset will process 4K/60 noticeably faster?

74. willywanta

Posts: 502; Member since: Jun 04, 2014

yeah. Just look at the lightning port.

3. basher

Posts: 105; Member since: Sep 20, 2015

first the 3.5mm jack port, now JPEG. What's next?

7. Panino

Posts: 62; Member since: Aug 13, 2015

screen or antenna

15. Rampage_Taco

Posts: 1049; Member since: Jan 17, 2017

they'll scrap the hard drive for personal storage and force users into the cloud.

21. TMHKR

Posts: 202; Member since: Dec 08, 2012

No force in this dimension of existence will make me move to cloud. Mass storage FTW!

38. Rampage_Taco

Posts: 1049; Member since: Jan 17, 2017

I don't mind cloud, but strictly as a backup. I'm happy with my physical storage, memory cards, and flash drives

31. adecvat

Posts: 639; Member since: Nov 15, 2013

You forgot front button.

44. whatev

Posts: 2218; Member since: Oct 28, 2015

First Adobe Flash, then CD/DVD, then Retina display, then NVMe, then 3.5 jack, now JPEG, what’s next? apple is so evil killing old technology, omg, someone do something and stop it!

4. Tipus

Posts: 856; Member since: Sep 30, 2016

More clusterf**k for people trying to exchange photos between apple, android and windows.

6. youssef44

Posts: 547; Member since: Apr 29, 2014

sounds good!

11. w1000i

Posts: 242; Member since: Jul 22, 2015

BS********

12. UglyFrank

Posts: 2193; Member since: Jan 23, 2014

The HEIF images do look a tiny bit softer in the cropped pictures but they look fine

13. HansP

Posts: 542; Member since: Oct 16, 2011

Considering wireless network speeds are getting into three digit megabit speeds and flagship phone storage starting at 64GB is becoming the norm, I honestly think they're doing it the wrong way. We need images closer to lossless, something like the PNG format, not a smaller file size, with even more compression noise, just to get yet another file format.

16. Rampage_Taco

Posts: 1049; Member since: Jan 17, 2017

RAW images should become a norm

43. HansP

Posts: 542; Member since: Oct 16, 2011

RAW is too much work for a casual photographer. :) Something like PNG is a good compromise. Get the image post-processing in-camera and get it all compressed into a file that doesn't kill detail or adds digital noise.

* Some comments have been hidden, because they don't meet the discussions rules.

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at https://www.parsintl.com/phonearena or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit https://www.parsintl.com/ for samples and additional information.