The OnePlus 5 cheats in benchmarks, just like those before it (Update - co-founder responds)

The OnePlus 5 cheats in benchmarks, just like those before it (Update - co-founder responds)

It looks like OnePlus is back to some of its old tricks with the new OnePlus 5, for XDA Developers caught the handset with a bit of virtual 'doping' during benchmarks. According to their report, which we have no reason to distrust as the board and attached news outlet bring together highly competent specialists, the OnePlus 5 review unit they have uses a mechanism to maximize the Snapdragon 835 SoC's performance during synthetic benchmarks. This is a problem because the smartphone isn't capable of sustaining the same performance level in regular daily use, hence users are being misled by what's really its theoretical peak power.

The mechanism in question is keeping four of the cores in the Snapdragon 835's octa-core CPU arrangement running at their maximum 1.9GHz frequency constantly, without the necesarry speed throttling that happens in accordance with sustained load. In its investigation, XDA discovered that when this mechanism is disabled, said cores run on their maximum 1.9GHz frequency only 24.4% of the time. Re-enabling it makes the cores hit 1.9GHz 95% of the time, which is a huge difference.

Thanks to this manipulation, which OnePlus explains is necessary to fully showcase the OnePlus 5's performance potential, the handset was able to achieve some of the highest scores commanded by a Snapdragon 835-powered device. However, it is dubious that these impressive numbers illustrate the actual performance in daily usage. The OnePlus 5 is still a very fast, modern smartphone, of course. But just like many athletes out there, it could do with a little more honesty about the true nature of its achievements.
Furthermore, benchmark cheating doesn't go especially well next to dropping support for the less than two years-old OnePlus 2, yearly price increase, and nabbing Apple's iPhone 7 Plus design when it comes to improving OnePlus' public image. Still, the OnePlus 5 is a compelling handset that's also reasonably priced. The strength of the offer should be able to carry the company forward for another year, at least.

The complete XDA Developers report is available in the links below if you wish to dig deeper into all the technological details of benchmark cheating. OnePlus isn't the only company to do so – the past remembers the likes of Samsung and HTC using this tactic too, hoping to squeeze a few thousand extra points out of AnTuTu or whatever. While synthetic benchmarks are great at providing a general idea of how a phone fares performance-wise, actual use is what really matters, and that's much harder to quantify.

UPDATE: OnePlus co-founder Carl Pei took to Reddit (link) to tackle the issue. His response, in short, is that the OnePlus 5 treats benchmarks and resource-intensive apps & games the same way. Hence, benchmark performance is supposed to equal real-life performance in those scenarios. However, XDA Developers identified a system specifically programmed to target the most popular benchmark applications in rotation. Hence, Mr. Pei's claims might not be as sincere as they come across. The full response is available below:



ALSO READ

Related phones

5
  • Display 5.5" 1080 x 1920 pixels
  • Camera 16 MP / 16 MP front
  • Processor Qualcomm Snapdragon 835, Octa-core, 2457 MHz
  • Storage 128 GB
  • Battery 3300 mAh

FEATURED VIDEO

83 Comments

1. doejon

Posts: 411; Member since: Jul 31, 2012

thats why benchmarks are for the ass

9. ahmuzencab

Posts: 192; Member since: May 12, 2017

I mean the benchmarks that come out of the ass are by the likes of Samsung and OnePlus, the ones who cheat to make their notsopowerful chip seem verysopowerful

14. lpratas

Posts: 398; Member since: Nov 09, 2011

That is a lie. Where is the prove that Samsung do cheating with their phones, normally nor even are the phones which have the best results in benchmarking.

29. trojan_horse

Posts: 5868; Member since: May 06, 2016

"Where is the prove that Samsung do cheating withtheir phones," Here is proof... from PA: http://www.phonearena.com/news/Samsung-Galaxy-Note-3-apparently-using-benchmark-booster-to-fix-results_id47886 And here is it from the original article at ArsTechnica:http://tinyurl.com/y6wxe3sc Samsung has been known to cheat on benchmarks too, bruh!

40. Cicero

Posts: 1101; Member since: Jan 22, 2014

You show an article posted in 2013. Long time has passed since then. And Samsung change it's attitude since then. Something actual you have?

51. Macready

Posts: 1792; Member since: Dec 08, 2014

... a long time ago, in a Galaxy far far away...

56. trojan_horse

Posts: 5868; Member since: May 06, 2016

"... a long time ago, in a Galaxy far far away..." @Macready That is why I said Samsung has BEEN KNOWN to cheat on benchmarks... I didn't say Samsung is still currently cheating on benchmarks. "HAS BEEN KNOWN" refers to the past, not to the present. My point was pretty clear in post #29.

81. Macready

Posts: 1792; Member since: Dec 08, 2014

Sure, but you were replying to posts talking in and about present time, not past perfect, let alone past tense. Hence my reply to highlight the difference between Samsung (now) and OnePlus (now).

68. yann

Posts: 604; Member since: Jul 15, 2010

Good one!

53. TechieXP1969

Posts: 14967; Member since: Sep 25, 2013

But they didn't cheat. They simply whitelisted the phone on tests to benchmark the cores at full clock. Cheating would be exceeding true clock speed. That is the whole point of a benchmark. If you were gonna race a car, you aren't gonna drive the speed limit on a racetrack...right? You're gonna push the car as fast as it was designed. Samsung didn't cheat. Nor did anyone else. Anyone who claims otherwise, doesn't understand what benchmark tools for.

64. ahmuzencab

Posts: 192; Member since: May 12, 2017

So every tech blogger, site, channel, etc was wrong.. except you? Yea sorry... no.

65. IT-Engineer

Posts: 452; Member since: Feb 26, 2015

Yes they are wrong. Being a tech blogger does'nt mean you understand software engineering, hardware, limitations and so on. I'm a software engineer and work as a system architect. What Oneplus and Samsung did is not cheating. They didn't tweak the benchmarking software to give false results. The hardware can run at 1.9 GHz underload, and so they did. There is nothing wrong, cheating or illegal.

73. adi90

Posts: 554; Member since: Dec 21, 2015

@IT Engineer and Techie If artificially boosting your benchmarks score is not cheating, then why samsung s8, ,mi mix , etc did not do the same thing which op5 did. You should know that the lower clocked cores ran at 1.9GHz for 95% of the time only beacuase it deducted that popular benchmarks is being run. Will they run for same duration of the time in other scenarios like 3d gaming even if a user wants ? And as far as testing the hardware power is concerned , why only op5 is so much eager to test the raw power of sd835. It means samsung and other manufacturers using sd835 do not care about their chip. The problem with you guys is that you think you are different than others. In that false sense you forget the difference between right and wrong.

71. omnitech

Posts: 1131; Member since: Sep 28, 2016

Of course they are wrong. They are Apple shills like you. If IOS was open, someone would have created an app to show that Apple probably does the same thing but probably in an even more shady manner. It's just that tech geeks cant do this on a closed system like they can do on Android.

78. ahmuzencab

Posts: 192; Member since: May 12, 2017

Attacking two companies makes me an "Apple shill" apparently. That's how I know your entire reasoning and any statement made from you is void/poor/false/a waste of time to read from now on. Good job.

55. TechieXP1969

Posts: 14967; Member since: Sep 25, 2013

That's where you are stupid. Because if the chip perform fast in the benchmark, that means that si what the hardware is capable of...you are such a dummy. Just because you may never run a chipset at full speed, doesn't mean the hardware isn't capable. Again you troll haters don't understand benchmark tool.

2. pixel_ftw

Posts: 83; Member since: Jun 19, 2017

Chinese and Korean companies are big cheater in the benchmarks app. Big score in the benchmark make those silly goose fanboys happy.

4. medtxa

Posts: 1655; Member since: Jun 02, 2014

lol very true they buy phone because benchmark number.

57. TechieXP1969

Posts: 14967; Member since: Sep 25, 2013

Really? Who you think buys a phone based only on benchmarks? You sound stupid and you have no facts to prove such. I mean, woudl you claim people buy cars based on the top speed on the speedometer? No you wouldnt claim that would you. Again you trolls are stupid. The people who blow this up, are simply pissed because their piss poor shi**y phone can't match the power of top device in similar benchmarks.

66. antifanboyism

Posts: 57; Member since: Jun 20, 2017

Benchmarks doesn't necessarily converts into real life performance. The proof of this is kirin 960 is evenly matched by snapdragon 835 . But it scores much less in antutu and other benchmarking . Even it scores are similar with so 820

15. lpratas

Posts: 398; Member since: Nov 09, 2011

Nor Samsung and neither and much less LG. That is a lie.

16. ngtitao

Posts: 10; Member since: Jan 13, 2015

Dude, you are generalizing base on an incident. We can't truly say all Americans are dumbass just because Trump is one right?

38. Leo_MC

Posts: 6090; Member since: Dec 02, 2011

Americans no, US citizens on the other hand...

3. adi90

Posts: 554; Member since: Dec 21, 2015

It seems Oneplus is copying not only iphone but also android OEM . Beacuse it Never Settles.

5. Tech-shake

Posts: 202; Member since: Nov 14, 2016

Phonearena is really going hard on oneplus this time around, jelly you didn't recive a review unit much??

19. ngtitao

Posts: 10; Member since: Jan 13, 2015

Phonearena is not the only one reporting this. It's every where in the web now.

6. HugoBarraCyanogenmod

Posts: 1408; Member since: Jul 06, 2014

"But but it's to show oneplus maximum potential so it's legit." Sounds like it's acceptable for athletes to dope in order to maximize potential. now that every OEMs are doping for benchmark, the result just like Korean beauty contest.

13. Awalker

Posts: 1956; Member since: Aug 15, 2013

Athletes don't dope because there's rules against it. There's no such rules with benchmarks. That's why reviewers tend to stay away from benchmarks.

17. HillaryClinton2020

Posts: 190; Member since: Feb 08, 2017

hahaha Korean beauty contest, their face looks so same i can be champion

20. lpratas

Posts: 398; Member since: Nov 09, 2011

You must have much to point to Koreans, for the case they did some bad things to you? Don't mix Koreans and Chinese people in the same bag.

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at https://www.parsintl.com/phonearena or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit https://www.parsintl.com/ for samples and additional information.