TouchWiz speed test: Does Samsung's interface really lag?
Intro
Does Samsung's Android-based TouchWiz interface lag? Given how the electronics giant is always at the forefront of technology and is rarely second to the finish line when it comes to packing the latest with its flagships, this might sound like an unfounded question. But the reality is that its proprietary TouchWiz skin is a controversial topic that has the community divided in two – those who worship Samsung for including as many extras and perks in it, and those that can't stand it. No middle ground – either black, or white.
But why are some among us so visibly dismissive, even mad, at TouchWiz? If you ask, they'll probably answer that it's simply a matter of responsiveness and speed of navigation – two qualities that, according to them, TouchWiz lacks. At PhoneArena, we're not neutral either, seeing as it's our job to evaluate devices that often happen to carry Samsung's name, and then feed those findings back to you. Objectively, our general feeling has always been that while improvements have taken place, TouchWiz still had a way to go before we could call it completely smooth. Stock Android smooth.
Trouble is, when you have no tangible proof other than your own experience to offer, it's usually hard work convincing Samsung loyalists that there's, indeed, a problem. The issue, then, is that an Android launcher is not something that you can easily benchmark. Quite frankly, until recently, we didn't think it possible at all. But it's also true that we've been after an appropriate methodology for a while, and a little over a week ago, our persistence paid off.
Methodology
So why is it so hard to benchmark an Android launcher? Put frankly, the tools simply don't exist. That is, there's no app dedicated to measuring the performance of the various aspects of a launcher – like the homescreen, lockscreen, app drawer, and the like – as most apps are generally confined to their own sandbox. But we came up with a way to do it, though we had to re-purpose two quite niche pieces of genius software to do it, and we weren't without help.
Back in the summer of '14, we blogged about a new benchmark entrant – GameBench – that claimed to be 'uncheatable'. To draw a parallel, just think of how a typical test of a computer GPU goes, and you're bound to think of frame rates (granted that you follow that stuff). GameBench does exactly that – it measures the frame rate of apps and games and then combines the median frame rate achieved by all same-model phones before arriving at a final performance score for that device. It addresses a fundamental flaw with synthetic benchmarks by making it considerably harder to game it, at least in broader terms.
But since Samsung's TouchWiz launcher isn't an app, we had to re-purpose GameBench so that it continues tracking frame rates even when simply navigating through the interface. Thankfully, the devs behind the tool were kind enough to do that for us, and voilà – we were in game.
At this point, some of you might be wondering what a frame rate is and why it's a valid way of benchmarking performance. For the sake of brevity, we won't go into specifics as far as the term is involved, and just plug this in: the more of it that you have, the smoother motion appears and the better your device is performing. That is, at lower frame rates (say, 15 frames per second, or FPS), even a layman would notice that the image is choppy, while a value like 60 FPS and above is considered ideal and near impossible for the eye to discern.
But we needed a baseline, as otherwise you'd be looking at a bunch of numbers that make insufficient sense on their own. And what better candidate than the one launcher that the geekier of us have sworn fealty to long ago – Google's stock Android. It is the cleanest, most bare-bones incarnation of Android that, anecdotes have it, is by far the most lightweight of all. But for this to be a valid comparison, we'd need to perform the exact same motions (scrolling, panning, etc) whilst using both, and that's too impossible a task for any human. Thankfully, an ingenious little app called RepetiTouch allowed us to do exactly that.
RepetiTouch is an automation tool – you record 'scripts' during which the app tracks things like taps and swipes. Since cross-device usage isn't supported, we had to reach out to the developer behind it, who was more than happy to convert said scripts so that they work on both devices. Sleeves rolled, we went ahead and recorded a number of scenarios that make up the typical (essential, even) kind of usage that your launcher and apps go through – scrolling on your homescreen and app drawer, navigating through apps like the Gallery and browser, and even replicating the use of your phone dialer. All of this with the idea that GameBench is tracking the entire time, keeping count of the frame rate.
Finally, we did some good ol' app load time testing and measured the time it takes the two devices (the Galaxy Note 4 for team TouchWiz, and the Nexus 6 for team Stock Android, both running the 5.0.1 Lollipop build) to open them under identical conditions. We did that by recording the tests on camera, and then splitting the frames of the footage to pinpoint the exact app launch time, and the precise moment it fully visualized on the screen.
Enough talk, it's results time!
Let's start off with the performance of both launchers in terms of frame rate. A few disclaimers first, though – keep in mind that the values in the table below will tell you more about the comparative performance of the two contestants than the exact frame rate you can expect from them to achieve (which is capped at 60 FPS, since their screens' refresh rate is 60Hz). That is because whenever the script loops, all motion stops for a second or so, and GameBench starts measuring 0 FPS, dragging down the median and the frame rate stability. Since the scripts are identical, the scores are still valid in relative terms.
Also, do keep in mind that median FPS is different than average FPS (read). As for the FPS stability tab, it simply indicates what percentage of all captured frame rates were identical to the mean. The lower the stability, the bigger the chance that you notice those small hiccups, as your device will have gone from a high frame rate to a lower one and back again, drawing your attention to the drop.
TouchWiz (Note 4) scores seen left, stock Android (Nexus 6) scores seen right. "HOT" means that the tested area had already been loaded into the device's memory, while "COLD" indicates that the test was performed after a reboot.
* All scrips were ran on both devices for approximately 150 seconds; higher is better
* Both devices were factory reset before running the tests
* Both devices had the same number of apps on-board, most of which identical
* Both devices had the same wallpaper and homescreen layout (widgets, icons, folders)
As you can see for yourself, while there are some exceptions in which the Note 4's TouchWiz software is up to snuff and rivals the performance of stock Android on the Nexus 6, the overall scores match our more subjective impressions during the test. Namely, all is well with TouchWiz when talking lighter operations, but when relatively heavier apps such as the Gallery and the built-in browser are involved, the image sometimes gets choppy and less fluid than on the Nexus 6.
Load time test
Next up, we'll look at how both devices performed in our load time test in which we measure how long it takes them to open and close an essential app. We broke down the test in 2 categories with 2 respective subcategories – cold/hot loads (app loaded for the first time since reboot/app loaded after it was cached into the memory) and the time needed to close them and return to your homescreen.
* All scores in milliseconds (ms), lower is better
* All scores have been rounded
We began this test almost assured that TouchWiz would struggle significantly to keep pace with stock Android, but after a while, it was obvious that Samsung has done some serious optimizations with the Android 5.0 Lollipop build of its UI. The results speak for themselves, of course, and though TouchWiz lost in the end, it's worth pointing out that when talking cold load speeds, it is actually 23% faster (overall) than stock Android across tested apps.
Unfortunately, cold loads are arguably an unimportant metric for the most part, as you usually have your apps loaded in the cache so that your phone can access them quicker. This (hot loads) is an area in which TouchWiz struggled visibly – compared to stock Android, TouchWiz loads apps 34% slower and is up to 58% lazier when exiting them. That's a non-trivial lead for stock Android, even though we're still talking about differences that are measured as fractions of a second.
Moral of the story
We suspect that Android 5.0 Lollipop had a lot to do with the Note 4's seemingly sudden boost in speed. We can't prove it though, as we don't have data from back in 2014, when it ran KitKat, though our gut tells us that Samsung did some good work with the latest build. Interestingly enough, the company is rumored to be working on slimming down TouchWiz with the Galaxy S6, which could push its highly controversial launcher into the hearts of more people. Whether that happens, however, is a question we'll only be able to answer come March 1st, when Samsung's next big thing is due to be announced.
FCC OKs Cingular\'s purchase of AT&T Wireless
Things that are NOT allowed: