How we rate products, evaluate cameras and test battery life
- How we review products and assign scores
- The testing protocol behind the PhoneArena Camera Score
- The protocol behind our PhoneArena Battery Score
- PhoneArena Q&A: awards, badges, is PhoneArena free to use?
1. PhoneArena Smartphone Review Rating explained
Our product reviews are always based on multiple days of real-world experience with each device. We absolutely never write reviews based solely on a specs sheet or a brief moment with a device.
Additionally, we run a number of benchmarks and tests for each device, such as our in-depth Camera Score benchmark. These benchmarks explore all the fundamental aspects of a given device and provide a firm, unbiased ground for comparison.
In our write-up, however, you should not expect a perfectly identical writing style for every review. Each reviewer is different and brings a unique, valuable and honest perspective, and we strive to keep that character.
At the same time, assigning a score based solely on personal experience can be erratic, and that is why we have introduced a new scoring system on PhoneArena, which has gone into effect for our smartphone reviews starting in late September 2023.
This new scoring system uses a weighted average of twelve fundamental categories of the smartphone experience, with carefully selected criteria for each of the categories. This ensures that all smartphones are treated equally and we can avoid personal bias in the score. On this page, we'll share details regarding our special review rating system, as well as our cutting-edge Camera Score benchmark for evaluating phone cameras.
Additionally, we run a number of benchmarks and tests for each device, such as our in-depth Camera Score benchmark. These benchmarks explore all the fundamental aspects of a given device and provide a firm, unbiased ground for comparison.
In our write-up, however, you should not expect a perfectly identical writing style for every review. Each reviewer is different and brings a unique, valuable and honest perspective, and we strive to keep that character.
At the same time, assigning a score based solely on personal experience can be erratic, and that is why we have introduced a new scoring system on PhoneArena, which has gone into effect for our smartphone reviews starting in late September 2023.
This new scoring system uses a weighted average of twelve fundamental categories of the smartphone experience, with carefully selected criteria for each of the categories. This ensures that all smartphones are treated equally and we can avoid personal bias in the score. On this page, we'll share details regarding our special review rating system, as well as our cutting-edge Camera Score benchmark for evaluating phone cameras.
These are the twelve fundamental categories that we rank separately to come up with our final score:
- Performance Daily (Speed and Performance for daily smartphone tasks)
- Performance Peak (Performance in heavy-duty tasks and gaming)
- Battery life
- Charging
- Wireless charging
- Photo Quality
- Video Quality
- Display Quality
- Biometrics
- Audio
- Software
- Design
Each of these categories carries a different weight in the scoring system, and some categories have a larger importance for the score than others. That is where we had to use our judgment and readers' opinions to find a balance that reflects the way most people use their phones. For example, the battery life of a device is a very important category that for most people carries more weight than the charging speed.
Is the price of a device accounted for in the score?
No.
This means that a budget phone will typically get a lower score than a higher-end device, since usually those low-end phones use slower or less capable components. Of course, there can be exceptions and some phones can punch way above their class. For example, the budget Pixel A series traditionally have had cameras that deliver nearly flagship-quality pictures.
However, as a rule of thumb, cheaper phones will get lower scores, while more expensive phones will receive higher scores, simply because they are typically faster, more capable and better.
This means that a budget phone will typically get a lower score than a higher-end device, since usually those low-end phones use slower or less capable components. Of course, there can be exceptions and some phones can punch way above their class. For example, the budget Pixel A series traditionally have had cameras that deliver nearly flagship-quality pictures.
However, as a rule of thumb, cheaper phones will get lower scores, while more expensive phones will receive higher scores, simply because they are typically faster, more capable and better.
Explaining each of the categories
Let us also guide you through the individual categories:
Category 1: Performance Daily
The perceived speed and smoothness for easy, daily smartphone tasks like calls, mails, browsing and social media
For this category, we rank the performance in daily tasks. Questions that we ask when we come up with a score in this category are the following:
- Do apps load fast?
- Is the scrolling smooth?
- How does the gesture navigation work?
- Is there any overheating with simple tasks?
Category 2: Extreme Performance
How well the phone performs for heavy duty tasks and gaming
This score is based on a weighted mix of benchmark scores, including CPU performance tests and GPU / gaming-related benchmarks.
Category 3: Battery life
How long does it last?
We perform three separate battery tests for each device. Using specialized equipment and software, we set each phone to a fixed brightness level to ensure an equal playing field and all tests are performed over a Wi-Fi connection.
Test #1: Web Browsing -- we run a script that cycles through a set of webpages and scrolls around at pre-set intervals, emulating a real world browsing session. This is our lightest battery test.
Test #2: Video Streaming -- we run the same video playlist over a Wi-Fi connection and at the same video quality at the full size of the screen, and measure how long the phones last.
Test #3: 3D Gaming -- we run a popular gaming title with the same graphics settings and in the same environment / level on each phone we test. This test stresses the GPU.
Based on those three results, we come up with a final battery score for this category.
Category 4: Charging speed
We measure and we rank
We use the recommended charger and the cable that comes in the box to measure how long the phone charges from 1% to 100%. Based on the results, each phone is assigned a score.
Category 5: Wireless charging speed
We test the wireless charging speed too!
We use the recommended wireless charger to measure how long the phone charges from 1% to 100%. Based on the results, each phone is assigned a score.
Category 6: Camera - Photo Quality
Assigning a score to an image is hard, but we have crafted a new system that translates complex image quality assessment into simpler numbers
We have created a comprehensive new system that looks at various aspects of image quality and we call it the PhoneArena Camera Score.
Under this new protocol, every phone goes through extensive testing, including all of its cameras across a number of different scenarios.
This process includes both a machine-run, computation element for precise calculation of deviation in exposure and color accuracy, as well as a manual inspection of images by our photography experts.
Here are some of the main criteria we evaluate in phone cameras:
- Exposure and subject exposure
- Color temperature and color tint
- Detail resolution level
- Oversharpening artifacts
- Portrait mode separation quality
- Preview reliability (HDR, resolution, portrait mode)
- Stabilization in video
- And more
We run the numbers and prioritize the more impactful elements of a camera system to come up with a separate score for each camera, so you have a score for the main camera, zoom quality, ultra-wide camera, and front camera.
Category 7: Camera - Video Quality
How good is the video quality in numbers
We use the foundation of our photography testing including color accuracy and exposure testing to assess video quality.
Of course, recording video comes with its own number of unique challenges, so we rate it differently and look at a bunch of categories like video stabilization that are unique to this format.
Category 8: Display Quality
We do a number of lab tests measuring various screen characteristics
We use professional equipment and software to measure the characteristics of a phone display: maximum brightness, minimum brightness, color accuracy, and more.
We use these measurements to come up with a score that shows how advanced the display on a phone is.
Category 9: Biometrics
Fingerprints and face ID performance ranked by performance and accuracyWe test the performance of a phone's fingerprint scanner or face recognition system for speed and accuracy, and assign a score based on the results.
Category 10: Audio Quality
Evaluating a phone's speaker system
We test the loudspeakers on a phone and assign a score based on how loud it plays, and the quality and richness of the sound. The classification is based on typical performance for smartphone loudspeakers.
Category 11: Software and Updates
The software update commitment matters
The defining factor in this category is the software update commitment for each phone and whether updates are expected to arrive in a timely fashion or with a delay. For example, a phone which is set to receive five years of OS updates will rank higher than a phone that will only be supported for three years.
Each phone is assigned a score based on that software commitment and update delivery speed, and some other factors like software restrictions are also considered.
Category 12: Design
Build quality, materials, looks
How well is this phone designed? That can be a subjective matter, but we have a set of criteria that help us come up with a rating based on the build quality, the materials used, any special design elements, and more.
2. PhoneArena Camera Score benchmark – how we evaluate cameras
In the ever-evolving landscape of smartphones, camera performance holds a pivotal position in shaping the overall user experience. Amidst a plethora of devices vying for attention, discerning the true photo and video capturing prowess of a smartphone can be a daunting task. To address this challenge, PhoneArena has introduced the PhoneArena Camera Score, a comprehensive benchmark that objectively assesses the imaging capabilities of smartphones across a diverse range of scenarios.
Objective evaluation of camera systems
Traditionally, smartphone camera testing has relied heavily on subjective evaluations, often leading to inconsistent and inconclusive coverage. Recognizing the need for a more objective and unbiased approach, we developed the PhoneArena Camera Score, a structured methodology that strives to objectively evaluate each camera on a smartphone based on a multitude of criteria.
Controlled test environment
To ensure consistent and reliable results, we’ve come up with a controlled test environment optimized to reliably test the exposure and color capabilities of each camera. This controlled setting isolates the performance of the camera hardware and software, eliminating external factors that could influence the evaluation, such as daytime, weather conditions, varying situations and so on.
It’s important to note that we’ve checked these controlled environment tests against performance in real-world scenarios numerous times and made sure that our test results are representative of the kind of image quality you’re bound to get once you go out there and start taking photos. On top of that, we continue to double-check against real-world performance and use real-world results to evaluate certain camera aspects, such as portrait mode separation quality.
At PhoneArena, we've had the chance to test and evaluate hundreds of phone cameras over the years, and we're confident the breadth of experience we've accumulated across our team of industry veterans and tech enthusiasts has been successfully integrated into this new camera evaluation process.
At PhoneArena, we've had the chance to test and evaluate hundreds of phone cameras over the years, and we're confident the breadth of experience we've accumulated across our team of industry veterans and tech enthusiasts has been successfully integrated into this new camera evaluation process.
How the PhoneArena Camera Score works
We evaluate each camera on a phone across a multitude of criteria: from essentials like exposure, color representation and detail resolution, to specialties like portrait mode separation quality and viewfinder reliability.
Depending on the camera (main, zoom, ultrawide or selfie), a bigger emphasis is put on those features seen as more foundational for the specific camera’s purpose. For example, the level of resolved detail has a higher relative weight when evaluating zoom cameras, as compared to when evaluating the ultrawide camera.
Here are some of the main criteria we evaluate in phone cameras:
- Exposure
- Subject exposure
- Color temperature
- Color tint
- Detail resolution level
- Oversharpening artifacts
- Portrait mode separation quality
- Preview reliability (HDR, resolution, portrait mode)
- Stabilization
Scoring system
Each test is evaluated individually, and the scores are weighted based on their importance for the specific camera type (main, zoom, ultrawide, or selfie). These weighted scores are then aggregated to produce a final Camera Score for the entire phone. This ensures that the Camera Score accurately reflects the overall camera performance of the device.
The PhoneArena Camera Score widget
Along with this brand new rating procedure, we’ve developed a special Camera Score widget, the purpose of which is to display all camera ratings for a particular phone model.
The widget will display an overall Camera Score rating, main sub-ratings for Photo Score and Video Score, as well as per-camera sub-ratings, so readers can get an idea of a particular camera’s showing.
The widget displays all these ratings, plus it provides indication of how a particular score compares relative to the best in each category. In the case of the examples above, you can see that the iPhone 15 Pro Max holds the current best score for Main camera in video, while its zoom photo score of 23 falls a bit short of the current best, which is 24 and belongs to the Pixel 8 Pro.
It’s important to note that we’ve assigned different weights to the different types of cameras (main, zoom, ultrawide and selfie). The ratios of the weights are as follows: 100 for the Main camera; 35 for Zoom; 30 for Ultrawide; and 35 for Selfie. These ratios are the same between the photo and video categories.
All the sub-ratings in the photo category make up the overall Photo Score, while the final Camera Score is the result of both Photo and Video scores, with the current weighting being 50:50 between Photo and Video.
It’s important to note that we’ve assigned different weights to the different types of cameras (main, zoom, ultrawide and selfie). The ratios of the weights are as follows: 100 for the Main camera; 35 for Zoom; 30 for Ultrawide; and 35 for Selfie. These ratios are the same between the photo and video categories.
All the sub-ratings in the photo category make up the overall Photo Score, while the final Camera Score is the result of both Photo and Video scores, with the current weighting being 50:50 between Photo and Video.
As smartphone camera technology continues to evolve, we remain committed to continuously refining our testing methodology to reflect the latest advancements and user habits. We regularly update our test scenarios, software, and evaluation criteria to ensure that the PhoneArena Camera Score remains the most comprehensive and accurate benchmark for smartphone camera performance.
The philosophy behind the PhoneArena Camera Score rating
Surveying the relatively uncharted territory of camera benchmarking, one question loomed large: what should we measure against? Because evaluating a camera, especially a smartphone camera, is inherently subjective. So, what constitutes a great phone camera? What is it, exactly, that we're looking to see from these snappers?
One plausible path, which we initially considered, was to establish "reality" as the benchmark. In this scenario, we would capture photographs of our test environment in a brutally realistic manner, closely mirroring how the human eye perceives it. We would then evaluate the images produced by the phone cameras against this lifelike reference.
While this approach holds merit, it encounters a significant hurdle: it contradicts the desires of users when it comes to their phone cameras. Users crave cameras that effortlessly generate stunning, shareable photos. With the exception of a few purists, most of us seek photos that enhance our appearance, elevate the beauty of our surroundings, and are instantly ready for sharing. Frankly, no one desires a dark, dreary, bluish photo simply because it was taken on an overcast day. If that's the case, we'd prefer our photos to be more balanced, warmer, and appealing.
While this approach holds merit, it encounters a significant hurdle: it contradicts the desires of users when it comes to their phone cameras. Users crave cameras that effortlessly generate stunning, shareable photos. With the exception of a few purists, most of us seek photos that enhance our appearance, elevate the beauty of our surroundings, and are instantly ready for sharing. Frankly, no one desires a dark, dreary, bluish photo simply because it was taken on an overcast day. If that's the case, we'd prefer our photos to be more balanced, warmer, and appealing.
We believe that phone manufacturers are currently overindulging in image processing. We've grown weary of the artificially sharpened aesthetics, the perpetually radiant and vibrant colors, and skin tones that render people with an orange hue.
However, we also understand that we can't dwell solely in the realm of make-believe. We cannot defy human nature and insist that phone cameras produce photos as realistic as possible because, in all honesty, that's not what we truly desire.
What we desire are photos that simply look great! And what constitutes "great"? Well, that's what phone manufacturers are constantly grappling with. Year after year, they tweak, revamp, innovate, refine, and upgrade, all in pursuit of a formula that resonates with customers.
Hence, we've chosen a less-traveled path, one that represents a middle ground. It's a road less traveled because we don't foresee reverting to a pursuit of realism in phone photography, but we also disdain the excessively processed appearance prevalent in many popular phones today. This is why we've set our benchmark target on what we call "Reality Plus." Essentially, it means we want our photos to remain grounded in reality, but have a tastefully enhanced look that makes them that much more desirable and suitable for viewing and sharing.
The PhoneArena Camera Score evaluates various objective aspects, such as a camera's ability to capture genuine detail, the reliability of its preview, or the extent of undesirable oversharpening.
However, when it comes to assessing the visual "style" of photos produced by our beloved little snappers, which has always been, and continues to be, an inherently subjective matter, we've chosen to base our ratings on our concept of "Reality Plus." This entails achieving a level of brightness and an approach to color reproduction that stays rooted in reality while tastefully enhancing a photo's visual appeal.
However, when it comes to assessing the visual "style" of photos produced by our beloved little snappers, which has always been, and continues to be, an inherently subjective matter, we've chosen to base our ratings on our concept of "Reality Plus." This entails achieving a level of brightness and an approach to color reproduction that stays rooted in reality while tastefully enhancing a photo's visual appeal.
3. PhoneArena Battery Score protocol
A staple of PhoneArena reviews for years, battery testing is an extremely valuable piece of our reviews.
We perform three battery tests in house for every phone we test:
- Test #1: Web Browsing
- Test #2: Video Streaming
- Test #3: 3D Gaming
Prior to running the tests, we ensure equal conditions are met on all phones: screens are set to the same brightness level using professional equipment for precision, and there are no notification-heavy apps to influence the scores. All tests are carried over a Wi-Fi connection.
The Web Browsing test is the lightest of the three. We run a script that cycles through a set of webpages and simulates scrolling at a set interval. We measure the time it takes to drain the battery under these conditions.
The Video Streaming test uses more resources. We play the same video playlist, using the same settings on the videos for consistency and measure the time it takes to drain the phone fully.
Third, the 3D Gaming test uses a real-life game in “easy” mode with carefully tuned settings for an equal playing field. This is the only test among the three to stress the GPU of a phone noticeably. Again, we measure the time it takes to drain the phone while running this game.
After all this testing, we are able to come up with a single number for battery life that we call the PhoneArena Battery Score. If you want to understand the battery life of a smartphone in just one number, this is it. Our PhoneArena Battery Score is roughly equivalent to real-world screen time.
You can find the battery tests in the corresponding section in PhoneArena reviews (usually towards the end of the review, in the battery section). There, you will also notice our Battery Rating Widget (it looks just like the above Galaxy S24 Ultra widget), which provides additional information such as where a phone ranks overall and for each individual test.
4. PhoneArena Q&A
Does PhoneArena give badges and awards?
Yes, we award a "Recommended" badge to all products which score an overall rating of 8 and up in our review section. The purpose of the badge is to signal to readers that the product in question is a remarkable performer in its category.
We also give out awards to deserving devices and manufacturers on a fairly regular basis. You can usually expect this during major industry events and exhibitions, like CES, MWC, and IFA.
We also love to give yearly awards at the end of each year, celebrating the best phones in different categories and across different price points.
Is PhoneArena free to use?
As you all know, every single feature on PhoneArena.com is accessible and absolutely free of charge to the visitors.
We do, however, allow paid advertisements in order to cover the expenses related to keeping the site up (hosting, staff, equipment, etc.).
You may also stumble upon affiliate offers while browsing PhoneArena. We may earn a commission if you make a purchase by following any of those links. Of course, any accompanying content remains unbiased and always strives to present the given product in the most objective manner possible.
Therefore, it should be clear that it is in our best interest to keep our visitors and attract new ones, by providing the most unbiased phone reviews as soon as a device is made available to us.