Top Mobile App Testing Devices and Operating Systems

Lauren Gilmore , Dog owner, expat, gin lover. Allegedly wise to the ways of digital marketing, PR, and social media. Currently waging a war on mediocrity in communication and storytelling.

July 16th, 2019

Mobile application testing is both time-consuming and expensive. Without it, though, you run the risk of building an app that’s buggy and full of errors.
The sheer number of devices and operating systems available pose a specific challenge for QA teams who must decide which mobile app testing devices to prioritize for their test coverage strategy.

Why real-device testing

It’s imperative you determine if you’re going to use a real device, an emulator, or a simulator. On the one hand, emulators and simulators can be faster and more efficient for your test coverage. However, they don’t allow you to run real-life and real-time scenarios.

While these tools are a great addition to any agile testing team, it’s still crucial to assess and understand the activity of your app on an actual device. For mobile device testing, this means fewer false positives and more complete, reliable coverage.
Additionally, there are specific situations where testing on a real device beats all other methods:

Testing for interruptions such as incoming calls and texts.
Testing the functionality of your mobile applications in various conditions such as bright sunlight, rain, and in both day and night settings.
Running a device test to ensure the application runs smoothly and the UI and UX are convenient for real users.
Stress testing including working for long, continuous hours and any battery issues that may arise.

Best operating systems to test on

There are a plethora of app options. And for good reason. With over five billion mobile users in the world, the number of app downloads in 2018 totaled 194 billion. In the first quarter of 2019, app users could choose from 2.6 million Android apps and 2.2 million iOS apps.

With the latest global stats from StatCounter, these are the top operating systems for testing:

iOS

12.3 (52.18%)
12.2 (20.03%)
12.1 (9.29%)
11.4 (3.8%)
10.3 (3.03%)
9.3 (2.41%)

And watch for Apple’s 13 beta version announced in June. Available today, it won’t be available for public consumption until September.

If you use an iPhone, you are well aware of the “Software Update, upgrade now or upgrade tonight?” notifications. iOS devices tend to skew towards the latest OS version. Android OS support has a different situation due to the fractured nature of models and manufacturers.

Android

9.0 Pie (18.61%)
8.1 Oreo (18.06%)
6.0.1 Marshmallow (15.16%)
8.0 (13.35%)
7.0 Nougat (10.34%)
5.1 Lollipop (9.42%)

Android Q is expected to be released in August 2019. Compatible Android Q beta phones include all Pixel phones to the OnePlus 6T.
It’s pretty cut and dry what operating systems you should test on then, right? Not necessarily.

With 50 percent of the total global app downloads last year, China stakes claim to the highest total number of downloads. In April 2019, Huawei confirmed it was building its own operating system. Now, this was in case Huawei lost its licensing agreement with Google on new Huawei Android devices… which it did back in May.

Considering Huawei has its own app store and with Android open source, it’s possible we’ll be talking about a third popular OS you should be testing next year. But that’s next year. So for now, we’ll still stick to Android and iOS.

Testlio QA Services Test on iOS and Android Platforms

Which mobile test devices to consider

There’s no denying the consistent popularity of the iPhone. In Q4 2018 alone, 46.89 million iPhones were sold worldwide. Additionally, there are over 2.3 billion Android phones currently being used around the globe.

Everyone knows that iOS downloads indicate an Apple device. However, thanks to the plethora of smartphones running on Android, it’s a bit more harrowing to figure out which device those users have.

Most app teams rely on analytics to track user actions and engagement. These analytics packages also track the device and operating system of users. You definitely want to adjust your device test coverage to match your users; as they may tend to skew one way or another.

For example, corporate executives tend to use the newest devices on the latest operating systems, while teenagers might be using phones that were handed down from their parents. You should skew your device selection based on your target audience, and even better, your actual audience (based on usage analytics).

Next, trends are important – if you have a photo or video app, then it may behoove you to test on a device with the best camera.
But trends are not the end-all, be-all and legacy devices and operating systems are as important as the latest gadgets. You should also consider tracking OS version analytics and support for different screen sizes.

Top 15 mobile app testing devices for Android and iPhone

iPhone 7

With almost 18 percent of the Apple smartphone market, the iPhone 7 dominated the second half of 2018.

iPhone 7 Plus

Holding its own, the iPhone 7 Plus – and its predecessor above – dominate 5.62 percent of Apple’s global market.

iPhone 6S

According to the same data, the second most popular model currently for 2019 is the iPhone 6s

iPhone X

With 12.7 million units sold, the iPhone X was the top-selling smartphone in the first 3 months of 2018 and ended the year with 11.57 percent of the market.

Google Pixel 3

Launched in Q4 of 2018, Google Pixel 3 has received rave reviews for its security, speed, and – of course – its camera. This was the first Google phone to break into the top 3 smartphones in the US market. It’s also a top 5 premium phone in Western Europe.

Samsung Galaxy S10

For the most discerning users, the Samsung Galaxy S10 – with a fantastic 6.4-inch display, the largest battery capabilities, and a triple-lens rear camera – is currently the best Samsung phone on the market.

iPhone XR

Not released until late October 2018, the iPhone XR ended the year claiming less than two percent of Apple’s market. That said, Greg Joswiak – Apple’s VP of product marketing – said it was the company’s most popular phone every day since.

iPhone XS and iPhone XS Max

Also released in late 2018, these two versions are both ahead of the iPhone XR in terms of users.

Samsung Galaxy S8 and S8 Plus

If global users are your goal, the Galaxy S8 and S8 Plus rank in the top 10 in most Western Europe countries. In fact, Samsung’s Galaxy series has consistently been a top contender globally.

Samsung Galaxy A5

Newer isn’t always better. The more affordable and just as popular Samsung Galaxy A5 is still widely praised for its performance and durability. At over 2 years old, it still maintains a top 10 position in most countries.

iPhone 8 and 8 Plus

With over a nine percent purchased rate each, Apple’s iPhone 8 and 8 Plus are still top-selling models.

Samsung Galaxy S9 and S9 Plus

Though the Galaxy S9 plus has 2GB more than its smaller counterpart, Android phones generally use RAM to keep multiple apps easily accessible in the background. Ranking in the top 10 most popular phones in the US and much of Western Europe, it’s a good decision to have these phones in your testing arsenal.

Samsung Galaxy Note 9

If you’re a mobile app game maker, the Samsung Galaxy Note 9 is your testing dream. With the highest-grade in DisplayMate’s screen test history, a 4000mAh battery, and liquid cooling, this phone can handle the most demanding mobile games.

Huawei P8 Lite

Having already overtaken Apple in market share in 2017, Huawei has been eyeing the position of the world’s second-largest smartphone vendor for a while. If you’re going to test on one of the Chinese maker’s devices – and you should – it should be this one as it currently holds steady as a top smartphone in 10 countries.

Xiaomi Mi 9

Xiaomi is one of the more popular Android device manufacturers. With one of the highest-rated cameras and a Tech Advisor award winner at MWC 2019, Mi 9 is one of the best phones in the world… and costs users a fraction of the price of its closest competitors.

These are just a fraction of the phones you could use to test your mobile app. If it seems daunting, Testlio’s mobile app testing experts have these devices in hand. Don’t approximate performance, contact us today and get 100 percent physical device coverage.