Rigorous Mobile Device Evaluation Methods Used

Kerry Wan/ZDNET

The way the smartphone fits into our daily lives has changed dramatically over the past decades, from being solely a communication device to now connecting us to the vast internet. Today, the definition of the smartphone is being altered again, with AI slowly but surely taking center stage in the mobile experience. It might even replace apps one day.

Also: The best phones to buy in 2024

No matter the outcome, the value of smartphones in modern society is immeasurable; it’s a must-have gadget. So, to help readers like you find the best handset for your needs and preferences, ZDNET’s team of mobile experts tests just about every phone that hits the market throughout the year, from Androids to iPhones . We even test the devices that claim they’ll replace smartphones .

If you’ve ever wondered how we evaluate the latest smartphones to decide if they’re worth recommending, here’s a breakdown of the various aspects we consider.

Disclaimer: This post includes affiliate links

If you click on a link and make a purchase, I may receive a commission at no extra cost to you.

How we test phones in 2024

Kerry Wan/ZDNET

For starters, the phones we review at ZDNET are mostly provided by manufacturers shortly before they launch to the public. That means our initial hands-on reviews are typically based on a week’s time (or longer) with the unreleased devices.

Within the embargoed time frame, ZDNET reviewers can test the latest features (ideally on the latest software patch), ask follow-up questions to manufacturers, and evaluate the devices without any influence from other reviewers. There are also moments when we purchase phones to test or review a device provided by a mobile carrier, not the manufacturer. In the latter case, we’ll explicitly credit the carrier in the coverage, though it’ll have no editorial influence.

While ZDNET primarily covers smartphone releases in the US market, we also evaluate international handsets to understand the competitive landscape better and have a frame of reference when making recommendations to international readers. We also attend trade shows, including CES and Mobile World Congress , to connect with industry experts and analysts.

What makes a phone ZDNET recommended?

For hands-on testing, five aspects determine whether or not a phone gets recommended : design, performance, cameras, battery life, and special features. The importance of each aspect will vary across users; some will value camera quality over battery life, and others just want a phone that’s unique and different. Generally, the order of importance is cameras, battery life, design, performance, and then special features.

To be included in our buying guides , the best smartphones must achieve above-average marks on all five criteria (with a reviewed score of over 3.5 out of 5), especially when compared to other devices priced similarly. Reviewers also consider the key differences between the latest phone models and their predecessors during the grading process.

Click here

Design and ergonomics

Kerry Wan/ZDNET

How a phone looks and feels can greatly influence the overall experience. There’s a reason why Apple stores meticulously arrange iPhones the way they do, with the most colorful options front and center. Does the latest A17 Pro processor on the iPhone 15 Pro Max truly matter if you’re already mesmerized by the adorably-sized, blue-colored iPhone 15 ? (Just me?)

But also, how does the phone feel when it’s tucked in your tight jeans or lightweight shorts? When testing and recommending phones, we consider design and ergonomics heavily, understanding that not everyone wants the biggest and most premium-feeling option out there. For example, a device with a plastic casing will serve you better than an all-glass build if you’re a construction worker or someone who’s often outdoors.

To truly test the real-world experience of using the latest iPhones and Androids, ZDNET reviewers often don’t accessorize the handsets with silicone or rubberized cases; instead, we browse, take pictures, and roam around with them as is. Phones get brownie points if they’re rated IP68, the industry standard for water and dust resistance.

Performance

Kerry Wan/ZDNET

Several factors affect a phone’s performance, including LTE/5G signal, battery life, and background tasks. Therefore, we typically begin our evaluations with a fully charged handset, all background tasks closed, and as stable of a mobile connection as possible. I’m based in New York City, so I typically test the performance capacity of phones across various signal areas, such as the subway (where LTE signal can range from poor to non-existent) and back home in Staten Island (where LTE signal is richer due to the lack of skyscrapers and congestion.)

Performance testing also includes putting phones through varying levels of graphic-intensive tasks, including importing and exporting spreadsheets, photo-editing in Adobe Lightroom, and playing mobile games like Genshin Impact and Asphalt 9. I’ll oftentimes have a music player app running in the background or YouTube Picture-in-Picture just to push the mobile processor a little more.

Of course, reviewers also consider the price of the tested devices, adjusting their standards and expectations accordingly.

Cameras

Kerry Wan/ZDNET

Arguably the most valuable aspect of today’s smartphones, built-in cameras have improved so much over the past few years that they’re now our most convenient (and reliable) tool to capture life’s most important moments. Testing phone cameras at ZDNET includes capturing hundreds of photos and videos of various subjects and in various lighting conditions. The list of subjects ranges from flower petals (for macro shots) to people (for portrait shots) to the moon (for zoom/periscope shots).

Also: The best camera phone of 2024: Expert tested and reviewed

Having a larger sample size to reference and compare with images from other phone models gives us the most accurate assessment of what phone camera is best at preserving details, colors, contrast, and more. Whether we’re evaluating the latest Samsung Galaxy phones to each other or with the latest iPhone , ZDNET reviewers can typically be found with more than one device in their pockets, both for comparison reasons and because we’re simply tech geeks.

Battery life and charging

Kerry Wan/ZDNET

It’s also important for us to evaluate how long phones last under light, moderate, and heavy usage, how long they take to recharge, and how they do it (wired, wireless, or both). We typically judge the endurance of phones based on screen-on time (SOT); that’s the total amount of time the screen is turned on, whether you’re scrolling through TikTok or typing an email. The higher the SOT, the longer the phone lasts.

On average, phones can score from three hours of SOT to upwards of nine hours of SOT, with the value resetting after 24 hours or when the phone is fully recharged. However, remember that a high SOT value is not always correlated to top-tier battery life; being able to play a Netflix video at full brightness for four hours straight is more impressive, endurance-wise, than leaving a text document on the screen for nine hours. Therefore, when speaking to the battery life of phones, we also describe it in a more practical sense – mentioning if a device can last one full day of usage, more or less.

Special features

Kerry Wan/ZDNET

Beyond the traditional testing pillars, we also consider phones’ unique and special features as we finalize our buying advice. Devices like the Samsung Galaxy S24 Ultra have a built-in S Pen stylus, the Nothing Phone 2a has a light-up back cover, and the OnePlus Open can fold and expand into a handheld tablet. Such features distinguish these devices from a bustling smartphone market, bringing added value to users. Of course, they’re judged by a practicality scale, and only the most useful gimmicks will earn our reviewers’ approval.