The most common handsets that are put to the benchmark tests are those running on Google’s highly popular Android platform. There is an abundance of benchmarking applications available in the Android Market, in which the most notable is called Quadrant. What benchmarking does is measure multiple performance aspects of the device and add the results together to get a final score. The higher the score your smartphone receives, the better.
As we saw years ago with the progression of PCs, the speed of the processor isn’t everything. Fortunately, manufacturers seem to be aware of this as evidenced by their advancements. There are many attributes that can affect the performance of a mobile device positively or negatively. A solid example is going back to the HTC EVO 3D benchmark that we did just a few weeks ago. While it may boast a 1.2GHz dual-core Qualcomm Snapdragon MSSM8660 processor to go along with a whopping 1GB of RAM and an Adreno 220 GPU, the EVO 3D was outperformed by other similar devices such as the Samsung Galaxy S II. But why?
One of the things that these tests do not take into consideration is the added user interfaces that manufacturers like to build on top of the Android platform. The HTC EVO 3D features the latest of HTC’s Sense UI which not only likes to eat away at your power but also occupies a good chunk of the device's memory. Given that memory is one of the main things measured by a benchmark test like Quadrant, it should be no surprise that the EVO 3D scores lower than the Galaxy SII which uses a somewhat less invasive user interface.
So what can you take from all of this? Can we trust benchmark results to resemble real world performance? The answer can actually be both yes and no. First of all, we have to be sure that there aren't any hardware/software incompatibilities between the device and the benchmarking app, which can be difficult to figure out, but suspiciously high or low results should lead to such thoughts. Then, it should be kept in mind that it's best to compare handsets with roughly the same system software installed, since it's obvious that custom solutions by manufacturers/developers may alter the benchmark results. That, however, doesn't mean that real-life performance will truly be different.
What do you all think? Are benchmark tests accurate means of testing the overall power of a device? Is it a good way to determine how well the phone will perform under real life conditions? We’d love to hear what you think on the subject so tell us what you think in the comments below!