How we rate products, evaluate cameras and test battery life


This is an explainer page on:
  1. How we review products and assign scores
  2. The testing protocol behind the PhoneArena Camera Score
  3. The protocol behind our PhoneArena Battery Score
  4. PhoneArena Q&A: awards, badges, is PhoneArena free to use?

1. PhoneArena Smartphone Review Rating explained


Our product reviews are always based on multiple days of real-world experience with each device. We absolutely never write reviews based solely on a specs sheet or a brief moment with a device.

Additionally, we run a number of benchmarks and tests for each device, such as our in-depth Camera Score benchmark. These benchmarks explore all the fundamental aspects of a given device and provide a firm, unbiased ground for comparison.

In our write-up, however, you should not expect a perfectly identical writing style for every review. Each reviewer is different and brings a unique, valuable and honest perspective, and we strive to keep that character.

At the same time, assigning a score based solely on personal experience can be erratic, and that is why we have introduced a new scoring system on PhoneArena, which has gone into effect for our smartphone reviews starting in late September 2023.

This new scoring system uses a weighted average of twelve fundamental categories of the smartphone experience, with carefully selected criteria for each of the categories. This ensures that all smartphones are treated equally and we can avoid personal bias in the score. On this page, we'll share details regarding our special review rating system, as well as our cutting-edge Camera Score benchmark for evaluating phone cameras.

These are the twelve fundamental categories that we rank separately to come up with our final score:

  1. Performance Daily (Speed and Performance for daily smartphone tasks)
  2. Performance Peak (Performance in heavy-duty tasks and gaming)
  3. Battery life
  4. Charging
  5. Wireless charging
  6. Photo Quality
  7. Video Quality
  8. Display Quality
  9. Biometrics
  10. Audio
  11. Software
  12. Design

For each category and for the final overall score, we use a 10-point grading scale, with “1” being the worst score and “10” -- the best.

Each of these categories carries a different weight in the scoring system, and some categories have a larger importance for the score than others. That is where we had to use our judgment and readers' opinions to find a balance that reflects the way most people use their phones. For example, the battery life of a device is a very important category that for most people carries more weight than the charging speed.

Is the price of a device accounted for in the score?


No.

This means that a budget phone will typically get a lower score than a higher-end device, since usually those low-end phones use slower or less capable components. Of course, there can be exceptions and some phones can punch way above their class. For example, the budget Pixel A series traditionally have had cameras that deliver nearly flagship-quality pictures.

However, as a rule of thumb, cheaper phones will get lower scores, while more expensive phones will receive higher scores, simply because they are typically faster, more capable and better.

Explaining each of the categories


Let us also guide you through the individual categories:

Category 1: Performance Daily

The perceived speed and smoothness for easy, daily smartphone tasks like calls, mails, browsing and social media

For this category, we rank the performance in daily tasks. Questions that we ask when we come up with a score in this category are the following:
  • Do apps load fast?
  • Is the scrolling smooth?
  • How does the gesture navigation work?
  • Is there any overheating with simple tasks?

Category 2: Extreme Performance

How well the phone performs for heavy duty tasks and gaming

This score is based on a weighted mix of benchmark scores, including CPU performance tests and GPU / gaming-related benchmarks.

Category 3: Battery life

How long does it last?

We perform three separate battery tests for each device. Using specialized equipment and software, we set each phone to a fixed brightness level to ensure an equal playing field and all tests are performed over a Wi-Fi connection.

Test #1: Web Browsing -- we run a script that cycles through a set of webpages and scrolls around at pre-set intervals, emulating a real world browsing session. This is our lightest battery test.
Test #2: Video Streaming -- we run the same video playlist over a Wi-Fi connection and at the same video quality at the full size of the screen, and measure how long the phones last.
Test #3: 3D Gaming -- we run a popular gaming title with the same graphics settings and in the same environment / level on each phone we test. This test stresses the GPU.

Based on those three results, we come up with a final battery score for this category.

Category 4: Charging speed

We measure and we rank

We use the recommended charger and the cable that comes in the box to measure how long the phone charges from 1% to 100%. Based on the results, each phone is assigned a score.

Category 5: Wireless charging speed

We test the wireless charging speed too!

We use the recommended wireless charger to measure how long the phone charges from 1% to 100%. Based on the results, each phone is assigned a score.

Category 6: Camera - Photo Quality

Assigning a score to an image is hard, but we have crafted a new system that translates complex image quality assessment into simpler numbers

We have created a comprehensive new system that looks at various aspects of image quality and we call it the PhoneArena Camera Score.

Under this new protocol, every phone goes through extensive testing, including all of its cameras across a number of different scenarios.

This process includes both a machine-run, computation element for precise calculation of deviation in exposure and color accuracy, as well as a manual inspection of images by our photography experts.

Here are some of the main criteria we evaluate in phone cameras:
  • Exposure and subject exposure
  • Color temperature and color tint
  • Detail resolution level
  • Oversharpening artifacts
  • Portrait mode separation quality
  • Preview reliability (HDR, resolution, portrait mode)
  • Stabilization in video
  • And more

We run the numbers and prioritize the more impactful elements of a camera system to come up with a separate score for each camera, so you have a score for the main camera, zoom quality, ultra-wide camera, and front camera.

Category 7: Camera - Video Quality

How good is the video quality in numbers

We use the foundation of our photography testing including color accuracy and exposure testing to assess video quality.

Of course, recording video comes with its own number of unique challenges, so we rate it differently and look at a bunch of categories like video stabilization that are unique to this format.

Category 8: Display Quality

We do a number of lab tests measuring various screen characteristics

We use professional equipment and software to measure the characteristics of a phone display: maximum brightness, minimum brightness, color accuracy, and more.

We use these measurements to come up with a score that shows how advanced the display on a phone is.

Category 9: Biometrics

Fingerprints and face ID performance ranked by performance and accuracy

We test the performance of a phone's fingerprint scanner or face recognition system for speed and accuracy, and assign a score based on the results.

Category 10: Audio Quality

Evaluating a phone's speaker system

We test the loudspeakers on a phone and assign a score based on how loud it plays, and the quality and richness of the sound. The classification is based on typical performance for smartphone loudspeakers.

Category 11: Software and Updates

The software update commitment matters

The defining factor in this category is the software update commitment for each phone and whether updates are expected to arrive in a timely fashion or with a delay. For example, a phone which is set to receive five years of OS updates will rank higher than a phone that will only be supported for three years.

Each phone is assigned a score based on that software commitment and update delivery speed, and some other factors like software restrictions are also considered.

Category 12: Design

Build quality, materials, looks

How well is this phone designed? That can be a subjective matter,  but we have a set of criteria that help us come up with a rating based on the build quality, the materials used, any special design elements, and more.


2. PhoneArena Camera Score benchmark – how we evaluate cameras


Apple iPhone 15 Pro Max
PhoneArena Camera Score
BEST 154
152
PhoneArena Photo Score
BEST 160
157
Main (wide)
BEST 85
82
Zoom
BEST 27
23
Ultra-wide
BEST 25
24
Selfie
BEST 30
27
PhoneArena Video Score
BEST 149
147
Main (wide)
BEST 79
77
Zoom
BEST 24
21
Ultra-wide
BEST 23
23
Selfie
BEST 28
27

In the ever-evolving landscape of smartphones, camera performance holds a pivotal position in shaping the overall user experience. Amidst a plethora of devices vying for attention, discerning the true photo and video capturing prowess of a smartphone can be a daunting task. To address this challenge, PhoneArena has introduced the PhoneArena Camera Score, a comprehensive benchmark that objectively assesses the imaging capabilities of smartphones across a diverse range of scenarios.

Objective evaluation of camera systems


Traditionally, smartphone camera testing has relied heavily on subjective evaluations, often leading to inconsistent and inconclusive coverage. Recognizing the need for a more objective and unbiased approach, we developed the PhoneArena Camera Score, a structured methodology that strives to objectively evaluate each camera on a smartphone based on a multitude of criteria.

Controlled test environment


To ensure consistent and reliable results, we’ve come up with a controlled test environment optimized to reliably test the exposure and color capabilities of each camera. This controlled setting isolates the performance of the camera hardware and software, eliminating external factors that could influence the evaluation, such as daytime, weather conditions, varying situations and so on.

It’s important to note that we’ve checked these controlled environment tests against performance in real-world scenarios numerous times and made sure that our test results are representative of the kind of image quality you’re bound to get once you go out there and start taking photos. On top of that, we continue to double-check against real-world performance and use real-world results to evaluate certain camera aspects, such as portrait mode separation quality.

At PhoneArena, we've had the chance to test and evaluate hundreds of phone cameras over the years, and we're confident the breadth of experience we've accumulated across our team of industry veterans and tech enthusiasts has been successfully integrated into this new camera evaluation process.

How the PhoneArena Camera Score works


We evaluate each camera on a phone across a multitude of criteria: from essentials like exposure, color representation and detail resolution, to specialties like portrait mode separation quality and viewfinder reliability.

Depending on the camera (main, zoom, ultrawide or selfie), a bigger emphasis is put on those features seen as more foundational for the specific camera’s purpose. For example, the level of resolved detail has a higher relative weight when evaluating zoom cameras, as compared to when evaluating the ultrawide camera.

Here are some of the main criteria we evaluate in phone cameras:

  • Exposure
  • Subject exposure
  • Color temperature
  • Color tint
  • Detail resolution level
  • Oversharpening artifacts
  • Portrait mode separation quality
  • Preview reliability (HDR, resolution, portrait mode)
  • Stabilization

Scoring system


Each test is evaluated individually, and the scores are weighted based on their importance for the specific camera type (main, zoom, ultrawide, or selfie). These weighted scores are then aggregated to produce a final Camera Score for the entire phone. This ensures that the Camera Score accurately reflects the overall camera performance of the device.


The PhoneArena Camera Score widget


Along with this brand new rating procedure, we’ve developed a special Camera Score widget, the purpose of which is to display all camera ratings for a particular phone model.

Apple iPhone 15 Pro Max
PhoneArena Camera Score
BEST 154
152
PhoneArena Photo Score
BEST 160
157
Main (wide)
BEST 85
82
Zoom
BEST 27
23
Ultra-wide
BEST 25
24
Selfie
BEST 30
27
PhoneArena Video Score
BEST 149
147
Main (wide)
BEST 79
77
Zoom
BEST 24
21
Ultra-wide
BEST 23
23
Selfie
BEST 28
27

Samsung Galaxy S23 Ultra
PhoneArena Camera Score
BEST 154
144
PhoneArena Photo Score
BEST 160
147
Main (wide)
BEST 85
77
Zoom
BEST 27
21
Ultra-wide
BEST 25
22
Selfie
BEST 30
27
PhoneArena Video Score
BEST 149
141
Main (wide)
BEST 79
75
Zoom
BEST 24
18
Ultra-wide
BEST 23
21
Selfie
BEST 28
26

Google Pixel 8 Pro
PhoneArena Camera Score
BEST 154
148
PhoneArena Photo Score
BEST 160
155
Main (wide)
BEST 85
79
Zoom
BEST 27
24
Ultra-wide
BEST 25
23
Selfie
BEST 30
28
PhoneArena Video Score
BEST 149
141
Main (wide)
BEST 79
77
Zoom
BEST 24
17
Ultra-wide
BEST 23
21
Selfie
BEST 28
26

The widget will display an overall Camera Score rating, main sub-ratings for Photo Score and Video Score, as well as per-camera sub-ratings, so readers can get an idea of a particular camera’s showing.

The widget displays all these ratings, plus it provides indication of how a particular score compares relative to the best in each category. In the case of the examples above, you can see that the iPhone 15 Pro Max holds the current best score for Main camera in video, while its zoom photo score of 23 falls a bit short of the current best, which is 24 and belongs to the Pixel 8 Pro.

It’s important to note that we’ve assigned different weights to the different types of cameras (main, zoom, ultrawide and selfie). The ratios of the weights are as follows: 100 for the Main camera; 35 for Zoom; 30 for Ultrawide; and 35 for Selfie. These ratios are the same between the photo and video categories.

All the sub-ratings in the photo category make up the overall Photo Score, while the final Camera Score is the result of both Photo and Video scores, with the current weighting being 50:50 between Photo and Video.

As smartphone camera technology continues to evolve, we remain committed to continuously refining our testing methodology to reflect the latest advancements and user habits. We regularly update our test scenarios, software, and evaluation criteria to ensure that the PhoneArena Camera Score remains the most comprehensive and accurate benchmark for smartphone camera performance.

The philosophy behind the PhoneArena Camera Score rating


Surveying the relatively uncharted territory of camera benchmarking, one question loomed large: what should we measure against? Because evaluating a camera, especially a smartphone camera, is inherently subjective. So, what constitutes a great phone camera? What is it, exactly, that we're looking to see from these snappers?

One plausible path, which we initially considered, was to establish "reality" as the benchmark. In this scenario, we would capture photographs of our test environment in a brutally realistic manner, closely mirroring how the human eye perceives it. We would then evaluate the images produced by the phone cameras against this lifelike reference.

While this approach holds merit, it encounters a significant hurdle: it contradicts the desires of users when it comes to their phone cameras. Users crave cameras that effortlessly generate stunning, shareable photos. With the exception of a few purists, most of us seek photos that enhance our appearance, elevate the beauty of our surroundings, and are instantly ready for sharing. Frankly, no one desires a dark, dreary, bluish photo simply because it was taken on an overcast day. If that's the case, we'd prefer our photos to be more balanced, warmer, and appealing.

We believe that phone manufacturers are currently overindulging in image processing. We've grown weary of the artificially sharpened aesthetics, the perpetually radiant and vibrant colors, and skin tones that render people with an orange hue.


However, we also understand that we can't dwell solely in the realm of make-believe. We cannot defy human nature and insist that phone cameras produce photos as realistic as possible because, in all honesty, that's not what we truly desire.

What we desire are photos that simply look great! And what constitutes "great"? Well, that's what phone manufacturers are constantly grappling with. Year after year, they tweak, revamp, innovate, refine, and upgrade, all in pursuit of a formula that resonates with customers.

Hence, we've chosen a less-traveled path, one that represents a middle ground. It's a road less traveled because we don't foresee reverting to a pursuit of realism in phone photography, but we also disdain the excessively processed appearance prevalent in many popular phones today. This is why we've set our benchmark target on what we call "Reality Plus." Essentially, it means we want our photos to remain grounded in reality, but have a tastefully enhanced look that makes them that much more desirable and suitable for viewing and sharing.


The PhoneArena Camera Score evaluates various objective aspects, such as a camera's ability to capture genuine detail, the reliability of its preview, or the extent of undesirable oversharpening.

However, when it comes to assessing the visual "style" of photos produced by our beloved little snappers, which has always been, and continues to be, an inherently subjective matter, we've chosen to base our ratings on our concept of "Reality Plus." This entails achieving a level of brightness and an approach to color reproduction that stays rooted in reality while tastefully enhancing a photo's visual appeal.


3. PhoneArena Battery Score protocol


Samsung Galaxy S24 Ultra
( 5000 mAh )
Samsung Galaxy S24 Ultra
Battery Life Estimate
6h 46m
Ranks #19 for phones tested in the past 2 years
Average is 6h 40m
Browsing
20h 6m
Average is 15h 10m
Video
8h 18m
Average is 10h 5m
Gaming
6h 28m
Average is 7h 53m
Charging speed
45W
Charger
68%
30 min
1h 9m
Full charge
Ranks #43 for phones released in the past 2 years
Wireless Charging
15W
Charger
33%
30 min
1h 58m
Full charge
Ranks #7 for phones released in the past 2 years


A staple of PhoneArena reviews for years, battery testing is an extremely valuable piece of our reviews.

We perform three battery tests in house for every phone we test:
  • Test #1: Web Browsing
  • Test #2: Video Streaming
  • Test #3: 3D Gaming

Prior to running the tests, we ensure equal conditions are met on all phones: screens are set to the same brightness level using professional equipment for precision, and there are no notification-heavy apps to influence the scores. All tests are carried over a Wi-Fi connection.

The Web Browsing test is the lightest of the three. We run a script that cycles through a set of webpages and simulates scrolling at a set interval. We measure the time it takes to drain the battery under these conditions.

The Video Streaming test uses more resources. We play the same video playlist, using the same settings on the videos for consistency and measure the time it takes to drain the phone fully.

Third, the 3D Gaming test uses a real-life game in “easy” mode with carefully tuned settings for an equal playing field. This is the only test among the three to stress the GPU of a phone noticeably. Again, we measure the time it takes to drain the phone while running this game.

After all this testing, we are able to come up with a single number for battery life that we call the PhoneArena Battery Score. If you want to understand the battery life of a smartphone in just one number, this is it. Our PhoneArena Battery Score is roughly equivalent to real-world screen time.

You can find the battery tests in the corresponding section in PhoneArena reviews (usually towards the end of the review, in the battery section). There, you will also notice our Battery Rating Widget (it looks just like the above Galaxy S24 Ultra widget), which provides additional information such as where a phone ranks overall and for each individual test.

4. PhoneArena Q&A

Does PhoneArena give badges and awards?


Yes, we award a "Recommended" badge to all products which score an overall rating of 8 and up in our review section. The purpose of the badge is to signal to readers that the product in question is a remarkable performer in its category.

We also give out awards to deserving devices and manufacturers on a fairly regular basis. You can usually expect this during major industry events and exhibitions, like CES, MWC, and IFA.

We also love to give yearly awards at the end of each year, celebrating the best phones in different categories and across different price points.

Is PhoneArena free to use?


As you all know, every single feature on PhoneArena.com is accessible and absolutely free of charge to the visitors.

We do, however, allow paid advertisements in order to cover the expenses related to keeping the site up (hosting, staff, equipment, etc.).

You may also stumble upon affiliate offers while browsing PhoneArena. We may earn a commission if you make a purchase by following any of those links. Of course, any accompanying content remains unbiased and always strives to present the given product in the most objective manner possible.

Therefore, it should be clear that it is in our best interest to keep our visitors and attract new ones, by providing the most unbiased phone reviews as soon as a device is made available to us.
FCC OKs Cingular\'s purchase of AT&T Wireless