x PhoneArena is hiring! Reviewer in the USA
  • Hidden picShow menu
  • Home
  • News
  • The numbers always lie: HTC, LG, and Asus all found to boost benchmark scores

The numbers always lie: HTC, LG, and Asus all found to boost benchmark scores

Posted: , by Michael H.

Tags :

The numbers always lie: HTC, LG, and Asus all found to boost benchmark scores
Samsung has made the news a couple times because the company has been caught artificially boosting benchmark scores on various devices, most recently with the Galaxy Note 3 which saw 20-50% increases in benchmark scores depending on the tests run. And now, more research from AnandTech is showing that almost no manufacturers are innocent in this practice, which really shouldn't come as a surprise.

AnandTech has been looking through the data since July, and has concluded that benchmark gaming is a pretty common occurrence. In fact, the only devices that were found to not change CPU behavior when a benchmarking app is detected were Nexus devices and newer Motorola devices (most pre-Google Moto devices weren't tested). The testing showed that at least one device from major manufacturers like HTC, LG, and Asus all push higher CPU frequencies in order to score better on benchmark tests. 

So far, Samsung seems to be the only manufacturer that has also changed GPU frequencies in order to game benchmark tests, and not all tests are being gamed. AnandTech talks about the need for manufacturers to simply stop the entire practice of altering test results, but that idea seems unlikely to us. Manufacturers all want to gain advantages, even if the number of users who would care about said advantages is relatively small. 

In reality, the most likely scenario is that benchmark makers and OEMs will continue to play cat-and-mouse. Benchmark makers will try to stay ahead of the optimizations, and OEMs will keep trying to gain artificial advantages. We can always try to rename every benchmark test, and get around the optimization whitelists. But really, the best option all around is just to stop putting all that much faith in the raw numbers (which were never that useful to begin with), and just look at real world performance. 

The numbers always lie: HTC, LG, and Asus all found to boost benchmark scores
source: AnandTech

61 Comments
  • Options
    Close






posted on 02 Oct 2013, 20:01 20

1. syedzain (Posts: 175; Member since: 08 Oct 2012)


Shame on u Android devices.. always tend to cheat.

We always knew iPhone 5s ( Apple Products ) were fast and smooth. No force close ...

iPhone FTW !!!!

posted on 02 Oct 2013, 20:08 35

3. willard12 (Posts: 1722; Member since: 04 Jul 2012)


Oh really?

http://www.phonearena.com/news/Which-platform-crashes-more-Android-or-iOS_id26542

posted on 02 Oct 2013, 20:51 6

16. ardent1 (Posts: 2000; Member since: 16 Apr 2011)


That was a dumb study because the ceteris paribus condition wasn't met. If you no controlled benchmark, then using absolute changes as opposed to relative changes, you are going to see biases.

It's not like people took college-level statistics.

posted on 02 Oct 2013, 23:15 7

35. Dr.Phil (Posts: 1330; Member since: 14 Feb 2011)


Of course it doesn't meet that requirement. We are talking about two different operating systems on several different pieces of hardware. However, the fact that that iOS crashed several more times than any Android device is something worth looking at considering that iOS standardizes their hardware whereas Android doesn't. So, how do we know that those crashes weren't on a device running a 512 mHz processor with 256 MB of RAM?

I agree with your notion that we can't accurately compare the two which is why I wouldn't use studies such as that to make a point. However, if you're going to argue with this guy over posting some study then you should also take an equal shot at the guy who said "We always knew iPhone 5s ( Apple Products ) were fast and smooth. No force close ... "

posted on 02 Oct 2013, 23:07

32. davenycept (Posts: 200; Member since: 03 Jul 2012)


Nice stat

posted on 02 Oct 2013, 20:16 4

6. Finalflash (Posts: 3218; Member since: 23 Jul 2013)


It is disappointing that this is being done and is reminiscent of the GPU wars on the PC as well as the CPU wars during AMD and Intels run at each other. They got nothing but marginal gains from this and the risk was serious publicity damage. Why would you even want to rig benchmarks when really, we are coming to a point where no one cares about how much horsepower your phone has that it will never use. Besides, they rigged the benchmarks that matter least, the ones that are just purely theoretical. In my experience, I just skim over those and look at the FPS charts from actual games to determine ability.

Tragically, this is a pretty one sided article and although I respect Anand and Brian for covering it so well, I would like to see what their methodology and results were when testing the iPhone and iPad for any optomizations/cheating Apple is doing. At the moment all we have is Anand's word that Apple does not do this as stated in the comments first page and in the Article where Apple, Google and Moto are cleared of any wrong doing. But really, the latter two were tested and shown the results off while Apple we are just supposed to take at their word. They really need show their work for those otherwise this just loses some serious credibility and seeming like a biased targeted test against Android OEMs. It would be harder to do I suppose because you cannot just rewrite the code a bit and install apps in iOS as easily, so methodology is very important and the results would then be verified based on how they tackled the problem.

posted on 02 Oct 2013, 21:24 1

25. Shatter (Posts: 2036; Member since: 29 May 2013)


Do not even half believe any benchmarks unless its a game or program like unzipping a 5Gb rar or something. Even games/programs are still faked sometimes so go a bit further and actually look at video comparisons by unbiased youtubers.

posted on 02 Oct 2013, 21:00 3

19. sprockkets (Posts: 1611; Member since: 16 Jan 2012)


Moto didn't cheat with the current crop of devices. Sony wasn't tested.

posted on 02 Oct 2013, 22:04 1

29. jacko1977 (Posts: 428; Member since: 11 Feb 2012)


No force close coz it dont tell u it just closes at lease with android u can send report to the dev
i bet ijunk as some cheating in software too
apple new theme
we steal we lie we cheat

posted on 03 Oct 2013, 01:35 2

43. boosook (Posts: 1437; Member since: 19 Nov 2012)


But I can't understand what's the problem in running your cpu at full speed in a benchmark… it seems logical to me. I want to score well, and I'm simply using all the power I have. Is this cheating?

posted on 03 Oct 2013, 13:25

58. theruleslawyer (Posts: 108; Member since: 23 Apr 2012)


It's not representing the performance the user will actually see on the device. Such a setting will trash battery life and possibly overheat and damage hardware in the long run.

posted on 03 Oct 2013, 04:51 1

50. Berzerk000 (Posts: 4275; Member since: 26 Jun 2011)


Notice how the Motorola and Nexus devices on the chart aren't boosting benchmarks, and those devices still beat the iPhone handily (aside from the 5S, but the Nexus 5 will fix that).

posted on 03 Oct 2013, 10:19

55. kabhijeet.16 (Posts: 811; Member since: 05 Dec 2012)


Arguments (TROLLS) before reading article...
Only Nexus / Moto devices are fair...
There is nothing mentioned about iPhone.....

posted on 18 Jan 2014, 05:39

61. MobileKicker (Posts: 212; Member since: 19 Sep 2013)


U said right syedzain iPhone FTW

And the android worm is proving otherwise

posted on 02 Oct 2013, 20:05 4

2. spiderpig2894 (Posts: 595; Member since: 10 Jan 2012)


So what about Sony? Could Sony be the only big phone manufacturer to be honest in benchmarks? Or are they refereing to almost all Android OEM?

posted on 02 Oct 2013, 20:11 3

5. Jason2k13 (Posts: 1217; Member since: 28 Mar 2013)


Probably all major OEM's

posted on 02 Oct 2013, 22:42 4

30. Taters (banned) (Posts: 6474; Member since: 28 Jan 2013)


Sony is the worst. LOL There is no way in hell the Xperia Z should destroy the Nexus 4 in benchmarks when it the Z has the same internals and pushes way more pixels.

posted on 03 Oct 2013, 00:00 5

36. dmn666 (Posts: 244; Member since: 19 Oct 2011)


Aww - little poop - pulling out more poop out of itself. Xperia Z doesn't cheat on benchmarks. N4 scores higher on GPU tests as it has lower res screens. Use google before you poop again

http://www.zdnet.com/benchmarks-htc-one-sony-xperia-z-googlelg-nexus-4-7000014067/

posted on 03 Oct 2013, 02:56 1

47. Taters (banned) (Posts: 6474; Member since: 28 Jan 2013)


Look at every other benchmark dumb ass. I can link Anandtech for you but i'll let you Google it yourself. The Xperia Z whips the Nexus 4 in most benchmarks even though technically it isn't possible with the same S4 pro chipset and a 1080p screen vs. a 720p screen.

posted on 03 Oct 2013, 12:22

57. Diazene (Posts: 129; Member since: 01 May 2013)


well, I recall the nexus 4 had throttling issues

posted on 02 Oct 2013, 20:19 4

7. ocilfa (Posts: 334; Member since: 03 Aug 2012)


If everyone is cheating, then isn't it not cheating?

posted on 02 Oct 2013, 20:45 7

12. GeekMovement (Posts: 2156; Member since: 09 Sep 2011)


No, if everyone is cheating, then it's everyone cheating duh.

posted on 03 Oct 2013, 07:09 1

53. Berzerk000 (Posts: 4275; Member since: 26 Jun 2011)


No, because everyone isn't cheating. Motorola and Google devices don't overclock themselves in benchmarks, as you can see in the chart in the article.

posted on 02 Oct 2013, 20:22 9

8. DukeX (Posts: 327; Member since: 28 Aug 2013)


Lmfao like i said the other day that other oems also do this. No one listened. And now this article pops up magically. Lol it's funny how they singled out Samsung. Freakin haters. No i don't have a samsung but fanboys will hate pantech,lg,sony,moto,zte,Huawei,oppo,the list goes on. Only because anyone of those could be the top oem and that's the reason why they hate! People only hate the one's who's winning the most!

posted on 02 Oct 2013, 23:07

33. HouTexan (Posts: 439; Member since: 08 Jan 2012)


Facts

posted on 02 Oct 2013, 20:30 3

9. GoBears (Posts: 451; Member since: 27 Apr 2012)


If they were really trying to "cheat" the consumers, I'm sure they would make it much more difficult to detect.

posted on 02 Oct 2013, 20:43 1

11. Scott93274 (Posts: 5030; Member since: 06 Aug 2013)


Well, I would hope that Motorolai's honest with its benchmark scores. I'm planning on getting a Droid Maxx around Black Friday if there are any deals to take advantage of. I got to play around with the phone today and I loved it.

posted on 02 Oct 2013, 21:08 4

22. DukeX (Posts: 327; Member since: 28 Aug 2013)


Tell yourself. Will those scores really matter if the phone runs smooth? Caringabout benchmark scores is like jumping for joy and freezing in midair. The joy is there but everyone will see you and be like "why is he so happy?"

posted on 02 Oct 2013, 20:45 2

13. xperiaDROID (banned) (Posts: 5629; Member since: 08 Mar 2013)


What about Sony, Huawei, ZTE, Xiaomi, Motorola and Oppo? Are these the only OEMs that are honest in benchmarks? I really hope so.

posted on 02 Oct 2013, 20:59 5

18. Jason2k13 (Posts: 1217; Member since: 28 Mar 2013)


no you dont, no one cares about benchmarks in 2013... i dont know about other OEM's, but samsung did not lie about benchmarks since they clearly advertised their phones to run at that full speed.

Samsung even mentioned games and most apps don't run at full potential like what the benchmark says, just to save battery which is perfectly fine, its not like we will see a difference in performance, but we will in battery life and overheating.

* Some comments have been hidden, because they don't meet the discussions rules.

Want to comment? Please login or register.

Latest stories