Samsung has made the news a couple times because the company has been caught artificially boosting benchmark scores on various devices, most recently with the Galaxy Note 3 which saw 20-50% increases in benchmark scores depending on the tests run. And now, more research from AnandTech is showing that almost no manufacturers are innocent in this practice, which really shouldn't come as a surprise.
AnandTech has been looking through the data since July, and has concluded that benchmark gaming is a pretty common occurrence. In fact, the only devices that were found to not change CPU behavior when a benchmarking app is detected were Nexus devices and newer Motorola devices (most pre-Google Moto devices weren't tested). The testing showed that at least one device from major manufacturers like HTC, LG, and Asus all push higher CPU frequencies in order to score better on benchmark tests.
So far, Samsung seems to be the only manufacturer that has also changed GPU frequencies in order to game benchmark tests, and not all tests are being gamed. AnandTech talks about the need for manufacturers to simply stop the entire practice of altering test results, but that idea seems unlikely to us. Manufacturers all want to gain advantages, even if the number of users who would care about said advantages is relatively small.
In reality, the most likely scenario is that benchmark makers and OEMs will continue to play cat-and-mouse. Benchmark makers will try to stay ahead of the optimizations, and OEMs will keep trying to gain artificial advantages. We can always try to rename every benchmark test, and get around the optimization whitelists. But really, the best option all around is just to stop putting all that much faith in the raw numbers (which were never that useful to begin with), and just look at real world performance.
Michael Heller is known for his clear and informative articles on mobile technology. He explores important topics such as the rollouts of 5G networks and the advancements in mobile payments, offering readers insights into the evolving tech landscape.
Recommended Stories
Loading Comments...
COMMENT
All comments need to comply with our
Community Guidelines
Phonearena comments rules
A discussion is a place, where people can voice their opinion, no matter if it
is positive, neutral or negative. However, when posting, one must stay true to the topic, and not just share some
random thoughts, which are not directly related to the matter.
Things that are NOT allowed:
Off-topic talk - you must stick to the subject of discussion
Offensive, hate speech - if you want to say something, say it politely
Spam/Advertisements - these posts are deleted
Multiple accounts - one person can have only one account
Impersonations and offensive nicknames - these accounts get banned
Moderation is done by humans. We try to be as objective as possible and moderate with zero bias. If you think a
post should be moderated - please, report it.
Have a question about the rules or why you have been moderated/limited/banned? Please,
contact us.
FCC OKs Cingular\'s purchase of AT&T Wireless
Things that are NOT allowed: