Is your device cheating on benchmarks? Here's how to find out
As time has gone by, so has people's reliance on benchmarks as a dependable and authentic tool for actual performance gauging. So much so, actually, that there are now those who would rather completely dismiss their usefulness (yes, they are useful, read on), and simply refuse to try and understand why it is that benchmarks are still exactly as relevant and exactly as dependable as they ever were. "How is that?", you may wonder, and that's a fair question, seeing as pretty much every major manufacturer has been caught red-handed trying to cheat their way to the top of the benchmark ladders. This has served to obfuscate the relative value of those tools. The truth is, benchmarks still have their role to play -- they're relatively rigid (i.e. standardized) in their approach, and a select few of them are actually pretty good at forecasting real-life results. In fact, if you spend your time benchmarking smartphones every other day, and then using them, it won't be at all that long until you start seeing the patterns that emerge. Are benchmarks foolproof? Heck no. They never were, and never will be. But they still play a role, and once you get used to their particular weaknesses and understand the infinite value of cross-benching, you start enjoying the fruit of your efforts and knowledge. Given enough practice, you'll start understating that these otherwise abstract scores should rarely be seen as absolutes, but rather -- as relative figures.
But how do you fight cheaters? Several apps that can monitor the behavior of your chipset are already available, and there are already available benchmarks that altogether avoid cheaters. We'll be showing you how to easily find out whether your particular device is cheating or not, and you needn't be a techie to make sense of this.
The free, but somewhat more burdensome (still dead easy) way to know if your device is cheating, is to install Qualcomm's Trepn Profiler app. The app is not available on the Google Play Store, but you can get it from HERE. Once you have it downloaded, simply use a microUSB cable to plug your device to the computer, and transfer the .apk file to your phone (destination doesn't matter, just memorize it). Then use either your built-in File Manager to locate it, or download one of the many found on the Play Store.By default, your phone will block non-Play Store installs of applications, disable the feature this time only.
Next up, we fire up Trepn Profiler and click the Profile System button, as shown. Once done, the app will start monitoring a few things by default: CPU0 and CPU1 Frequency, which are the two cores of the Samsung Galaxy Core Plus used in this tutorial (it runs a Broadcom chip). As you can probably already see, the graph will look jagged, as the chipset will try to its best to conserve core by altering the CPU frequency, depending on the work load. You can experiment sliding around to get a feel for it.
Enough talking, lets see if this mid-range Samsung phone cheats on benchmarks. It's as easy as having Trepn Profiler run in the profile and then fire up a benchmark of your choice. We tested out our Sammy device with 3 popular benchmarking tools -- AnTuTu, 3DMark and Quadrant. As you can see from the images, the phone cheats on all of them, raising its clock to the max. available frequency as soon as the app is detected in the background. We haven't even began the test. This kind of behavior can be replicated on a multitude of other devices, including ones from Asus, LG, HTC, Sony, Samsung. Devices from Motorola, along with the Google Nexus line, do not cheat as of yet, at least to our knowledge.
Got a quad- or eight-core smartphone, but don't know how to get an overlay for them? Easy enough, just go to Trepn Profiler's Settings, go to Overlays and select a blank overlay box. Once there, select Type: Graph and add the variable you wish to track (e.g. CPU2/CPU3 Frequency (core 3 and core 4, respectively).
We took a look at how the LG G2 reacts to a few popular benchmarks, and sure enough found that it cheats on AnTuTu and Vellamo. Interestingly, LG hasn't tweaked its system to spoof 3DMark. Again, benchmark not running, but all four cores shot up to their maximum operating frequency -- 2.3GHz.
If sideloading .apk files is too annoying for you, and you don't mind spending $3.99 for an app, you can grab System Monitor right from the Google Play Store. It works in a very similar way to Qualcomm's Trepn Profiler, and requires no extra setup beyond activating the overlay graphs like show on your left.Simply go to Settings > Flying Monitor > enable Monitor > go hunt some cheaters.