Qualcomm backing development of a new mobile benchmark standard

Qualcomm backing development of a new mobile benchmark standard
In light of allegations that some manufacturers are fabricating benchmark tests to make their gear look and smell like a rose when in reality it might actually smell like something else, Qualcomm is backing an effort by Berkeley Design Technology Inc., BDTI, to create new and independent user experience rating scheme for mobile devices.

There have been some assertions that Antutu favors Intel based architecture, so this backing by Qualcomm is arguably not without an agenda.

According to BDTI, "Many existing mobile benchmarks do not measure attributes relevant to consumers and can be easily manipulated. There is a striking lack of mobile benchmarks that measure actual user experience. Our user-experience-focused tests will include real-world measurements of application performance, battery life, and data transfer speed and delay."

The new standard will be designed solely with mobile technology in mind and the benchmarks will note common scenarios rather than complex equations. Qualcomm states this solution is “timely.”

BDTI will have an initial public device results report available later this year and there will be general availability in early 2014.

source: Qualcomm

FEATURED VIDEO

13 Comments

1. noler

Posts: 326; Member since: Aug 19, 2013

"some manufacturers are fabricating benchmark tests to make their gear look and smell like a rose when in reality it might actually smell like something else" Shame on you Samsung!

2. Shatter

Posts: 2036; Member since: May 29, 2013

Samsung auto overclocks the cpu when needed so it will still have an edge. It is not a benchmark fake it works in every app that needs extra power.

3. Igneel

Posts: 19; Member since: Jul 23, 2013

If I'm not mistaken, it overclocks on 'certain' benchmarks only.. So yeah. Not every app that needs extra power

5. Dr.Phil

Posts: 2487; Member since: Feb 14, 2011

You are somewhat mistaken as it also overclocks on most of their standard applications such as the web browser, video player, and other full-screen Samsung applications.

6. Ninetysix

Posts: 2965; Member since: Oct 08, 2012

No Dr.Phil. Only the whitelist apps will run at 532/533. if the app is not in the whitelist, it runs at 480. http://www.anandtech.com/show/7187/looking-at-cpugpu-benchmark-optimizations-galaxy-s-4

7. jdoee100

Posts: 334; Member since: Jun 04, 2013

I did not read the whole anandtech article, but this is from the verge concerning the same article, "but the site(Anandtech) did note that other benchmark apps that are not explicitly mentioned in code were also behaving the same way." So this means, other programs do run at higher speeds, not just the whitelist.

8. special4u

Posts: 63; Member since: May 22, 2013

samsung foreseen oneday their lies will broke, so they put in others "application" to argue in future. lol web browser?? video player?? full screen samsung application?? why not games & others calculate intensive apps????

14. DannydaKid

Posts: 6; Member since: May 15, 2013

Yeah, why not some graphic benchmark either, what Samsung did is letting the GPU run at the "designated clock" for a "Overall Benchmark", not a "GPU Benchmark" such as GLBench . They must be very stupid as well, and nice sacarsm :lol:

10. joey_sfb

Posts: 6794; Member since: Mar 29, 2012

Its very common in the PC industries to do this, ATI and Nvidia has been doing this for years and accuse the other parties for it. What i see is mobile performance is getting more and more attention and is good for me as i like to play a lot of mobile games.

4. james004

Posts: 486; Member since: May 15, 2013

then this comes down to the developers of benchmarks and their honesty. people believe in those apps and developers should respect it. update the apps to standardize the platform and avoid false benchmark reports that OEMs might be trying to cheat into.

9. kozza3

Posts: 778; Member since: Oct 17, 2012

how hard is is to make a benchmark that can't be abused?

13. zennacko unregistered

Not hard at all when you're not a part of industry (so they can't tell you're somehow biased towards X or Y manufacturers). But then they'll tell you "clearly have a lack of expertise in the industry (because you're not a part of it) and market, thus your benchmark results could be fake and by all means unreliable" Let's see how fair this Qualcomm-supported benchmark will be towards Intel, Samsung, MediaTek and other manufacturers...

12. marbovo

Posts: 658; Member since: May 16, 2013

What is needed is an app that colect data on a period of time, like hours or even days not just on the app execution, I think that would be very hard to cheat

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at https://www.parsintl.com/phonearena or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit https://www.parsintl.com/ for samples and additional information.