Huawei Ascend P7 found to be cheating – Futuremark disqualifies it
Samsung, LG, HTC, and Asus have been found guilty of cheating on benchmarks before, and now – Huawei gets a spot in the not-so-honorable list. Namely – the Ascend P7 has been delisted from Futuremark's website and its results have been stripped. The reason is that it's been discovered that the device does not act completely honestly, when in the presence of a benchmarker, such as GFXBench T-Rex and 3DMark Ice Storm.
Tech blog AnandTech did some testing by running the two apps on an Ascend P7, then renaming them and running them again. Surprise, surprise – the results were noticeably different. T-Rex gave the phone a 12.3 result, when ran directly from the Play Store; after renaming, it dropped the results to 10.6. That's a 16% boost in performance, courtesy of benchmark cheating. The results are a bit more drastic with Ice Storm – with 7462 points with the stock name and 5816 when renamed, that's a 28.3% difference.
The Ascend P7 is powered by a home-brewed Kirin 910T and a Mali-450 GPU. It is a far cry from the Snapdragon 800 and it wasn't wowing us with amazing results when it hit the market, but we still enjoyed the performance of the P7, so it is kind of disappointing to see Huawei showing such lack of trust in their own device.
1. CPU workload with stock T-Rex
2. CPU workload with renamed T-Rex
3. Cores used with stock T-Rex
4. Cores used with renamed T-Rex
source: AnandTech via Android Police
3. xperiaDROID (limited) (Posts: 5399; Member since: 08 Mar 2013)
Even Huawei cheated on benchmarks, does this mean that Sony is the only one that's being a good boy and playing fair in the Android world?
I love you, Sony!
6. rency0722 (Posts: 60; Member since: 12 Jun 2013)
Is that the reason why Sony keeps opting on lower clocked chipsets?
9. Taters (Posts: 2824; Member since: 28 Jan 2013)
You are hilarious. Lol
First this isn't cheating. It is an Apple propanda machine as an excuse of why iPhones get destroyed in benchmarks. The benchmarks on these phones are pretty indicative of what real performance of the chipset will be on a tablet with a much bigger battery and less thermal throttling. Basically means if the chipset can do it, it's not cheating.
Second, the biggest discrepancy between two devices using the same chipset is, you guessed, a Sony device. The Nexus 4 and Xperia Z use the EXACT same chipset, ram, glass back thermals, everything almost. Yet the Xperia Z bitch slaps the Nexus 4 in benchmarks yet performs considerably poorer than it in real life tests. So if anything, Sony is the ultimate cheater if you choose to ignore my first point.
11. Arte-8800 (limited) (Posts: 4281; Member since: 13 Mar 2014)
Nexus is vanilla, simple featureless phone and skin. Sony skin has features.
That all depends on Sony engineering, on how the Cpu are floating and idling.
16. Taters (Posts: 2824; Member since: 28 Jan 2013)
Software makes a difference but all s600 devices and s800 devices had similar performance benchmarks. The z was the only s4 pro device that benched ridiculously higher than the other s4 pro devices like the nexus 4 and LG optimus g. The LG optimus g skin was much heavier than both the Xperia z and nexus 4 yet somehow the z completely annhilates the optimus g and nexus 4.
Like I said. If it was just software tweaks doing these discrepencies then the Xperia z2 would destroy the GS5 and M8 in benchmarks yet itdoesn't.
Therefore they all cheated with the Z2 generation and only Sony cheated with the Xperia z generation. Simple logic.
Therfore Sony is the ultimate cheater or the cheater pioneer.
4. g2a5b0e (Posts: 2157; Member since: 08 Jun 2012)
A device running at full steam during a benchmark in order to show what it is fully capable of. I'm so tired of this being called cheating. That argument is played. More Phone Arena bullsh*t. Move along, folks. Nothing to see here.
5. Ashoaib (Posts: 1837; Member since: 15 Nov 2013)
lolz... you are usually against PA.... why? :D
13. g2a5b0e (Posts: 2157; Member since: 08 Jun 2012)
It's a very recent thing. Believe me. I just feel as though the quality of reporting on this site has gone down significantly over the last few months.
20. PapaSmurf (Posts: 8140; Member since: 14 May 2012)
Don't even get me started on the false numbers they report when they test the displays.
8. Japultra (Posts: 23; Member since: 16 May 2014)
I completely agree with you. It would be unrepresentative if the CPU DIDN'T reach its potential maximum. I mean, isn't that exactly what we're trying to figure out? Any other measure would be moot as we don't know what's running on the phone.
10. Taters (Posts: 2824; Member since: 28 Jan 2013)
Exactly. It seems almost like a propaganda machine started by ifans to discredit high end Android phones.
17. Berzerk000 (Posts: 3989; Member since: 26 Jun 2011)
This really is getting stupid. Cheating would be if the device overclocked itself while the benchmark was running, not this.
23. fzacek (Posts: 1580; Member since: 26 Jan 2014)
The processor is artificially held at its maximum, which is not representative of the phone's actual performance during normal use...
28. Taters (Posts: 2824; Member since: 28 Jan 2013)
Who made up this artificial gauge that benchmarks test regular use? There are much better ways to gauge regular performance, like using your eyes and brain.
Benchmarks should absolutely test peak theoretical performance. Why else would you run benchmarks? It's all a big dick waving contest. And just because your dick might perform better in regular use like peeing, it's still a smaller dick than the benchmark dick.
7. FoneAddict (Posts: 202; Member since: 05 Jul 2011)
Shambolic really, this industry is like the Wild West!
12. Deaconclgi (Posts: 258; Member since: 03 Nov 2012)
Hmmmm....seems like companies and most PA readers don't understand what the CPU is supposed to do during a benchmark.
So....its ok for my i5 4xxxK and GT 780 OC ACX to run at max speed on a benchmark but its called cheating when my phone's CPU and GPU run at max speed to see what the components benchmark at?
To me I sounds like the problem is with Android not having universal benchmark code that enables the benchmark program to have all phones, cpus/gpus run at their hardcoded max without OEMs have code in their Android build that kicks the CPU/GPU into high gear.
The benchmark application should cause the CPU/GPU to run at their maximum clock speeds. Maybe Android needs standard power management settings and one of them being Unrestricted or Managed By Application therefore the benchmark app can actually benchmark the components freely.
21. TylerGrunter (Posts: 905; Member since: 16 Feb 2012)
In my opinion it´s you who don´t understand what´s the problem here.
Benchmarks don´t make CPUs or GPUs run at their full speed, among other things because that would hit their termal limits and would reduce the life span of the SoC.
It is cheating because this kind of performance is ONLY allowed for the benchmark applications, meaning you will never get that performance in real life because the SoC will try to save battery.
In your own words: your i5 4xxxK won´t behave different in the benchmark than it will in other situation, so it´s not cheating. if Intel decided to let it run FASTER during benchmarks, then and only then it would be cheating. Which is the case here.
25. Taters (Posts: 2824; Member since: 28 Jan 2013)
Actually you do get the benchmark level of performance on some highly busy levels of some games, so there goes your whole argument.
Besides, if the benchmark was meant to test the chip, then that is still valid. The chip doesn't have to live by the constraints of the phone. In a tablet or a laptop it can probably run those numbers just fine without harming the CPU/GPU.
Also, do you really think some companies would risk a phone catching on fire or a warranty issue by pushing past the CPU limits for any app, including benchmark apps? Um no....
The benchmark is an app, so if it can run that app on that level,MIT can run any app at that level if they really wanted to. How is that cheating? Jeesus.
14. g2a5b0e (Posts: 2157; Member since: 08 Jun 2012)
This article is just more garbage click bait from Phone Arena. Calling this cheating is akin to telling me to run a race, then when I gather all my strength & try my absolute hardest to win, calling me a cheater. Makes absolutely no sense.
22. TylerGrunter (Posts: 905; Member since: 16 Feb 2012)
Your analogy is wrong. This is like having a 100m race, everyone starts from stand by position and you decide that to you they have to start counting the time when you are already at top speed because you want to have a better time than the rest.
Will they allow you to do that? Because that´s exactly what this cheat does.
26. Taters (Posts: 2824; Member since: 28 Jan 2013)
Extremely poor analogy. Because a benchmark measures just your top speed not how long it takes you to get to top speed.
If you were to gauge how long it takes you to get to top speed, you would use a stop watch, not a benchmark. Lol
31. g2a5b0e (Posts: 2157; Member since: 08 Jun 2012)
No, your analogy is wrong & Taters' explanation is spot on. This is not cheating. Plain & simple. It's a benchmark test. Benchmark. A standard by which all others are to be measured. It only makes perfect sense to measure what these values would be at their absolute best. Otherwise, who gives a sh*t? No one cares what the world's strongest man can lift when he's not putting forth his best effort.
33. aksa123 (Posts: 195; Member since: 30 Jan 2014)
It violated the futuremark's rules, thus cheating.
It's like using a drug in a race : you can run at full speed for a long time without getting exhausted. Sure, you can attain max speed, but that's not the point here. He's found cheating, thus disqualified (delisted).
34. g2a5b0e (Posts: 2157; Member since: 08 Jun 2012)
There's not much you can say dissuade my thoughts on this. Thanks though.
15. josephnero (Posts: 305; Member since: 16 Nov 2011)
this is called cheating because in real life usage when using tons of apps and super heavy games the CPU never hits that mark.only it lags
18. Taters (Posts: 2824; Member since: 28 Jan 2013)
It's only cheating if benchmarks were designed to gauge real life performance on a smartphone.
If it is designed to test the max theoretical power on any device, which is the defintion of a benchmark. Then it's not cheating.
19. Bondurant (Posts: 54; Member since: 04 Jun 2014)
The Huawei phones have internal process to identify app and selectively boost performance depending on the type of app like gaming. So possible that the tester does not know this and thinks it cheating.
27. Taters (Posts: 2824; Member since: 28 Jan 2013)
I believe the definition of that is software performance tweaking.
30. paul.k (Posts: 18; Member since: 17 Jul 2014)
As per Futuremark's Benchmark Rules and Guidelines:
"Benchmark specific optimizations are not allowed. Additionally, all generic optimizations that change the work specified by the benchmark are prohibited."
"The platform may not detect the launch of the benchmark executable."
Breaking or bending a rule in order to achieve a higher result is the textbook example of "cheating", thus - the choice of words
35. Japultra (Posts: 23; Member since: 16 May 2014)
Maybe the definition needs to be changed. It seems pretty sensical that a test used to measure performance potential should actually measure performance potential - which means the max the hardware configuration can achieve.
32. boidsonly (Posts: 6; Member since: 14 Dec 2012)
I'm shocked-the Chinese caught lying, cheating and stealing?!