How to Select the Right Set of Devices for Mobile Testing?

Mobile phones have been a great revolution of mankind. Big static land phones have slowly got obsolete and mobile phones are now an indispensable part of our everyday life.  In today’s world, almost every person owns, at least, one phone and a few have more than one. The smart phones’ entry into the market have made people go crazy and it is the gadget in which people are so dependent that without it they would feel lost. It is now very evident for almost all the industries that it is easy to reach customers via apps. Most of the mobile solutions are dependent on the new age disruptive technologies. Compared to other computing devices the reach of mobile/ smart phones is huge. With its accessibility/ availability to the high profile business magnets to a road side tea seller, mobile phones have filled up the so- called “digital divide” to a larger extent.

Challenge
From its traditional role as a mere communication tool, mobile phones have now become multipurpose gadgets used for both personal and professional uses, which creates opportunities as well as challenges like both sides of a coin. Technology shifts, proliferation of devices/ operating systems are creating challenges for hardware manufacturers and application developers in terms of developing and rolling out new products or updating it. The mobile application testing across various devices and platforms has now become even more challenging. As there are quite a lot of mobile makers in the market it is almost impossible to ensure that proper mobile testing is done on all the devices, and to a certain extent, it is not required as well. Digital modernization has encouraged people of any age group to manage their important data or images in the cloud, rely on apps that can work as reminders, use messengers to keep in touch and many more. As the wants have now become basic needs, one of the key areas that should be in focus is the Customer Experience. The user’s geo, age group, and the targeted group of customer’s info is crucial in deriving the best customer experience the user can have. However, yet another point to be remembered is that in the competitive mobility  arena ‘go to market’ time has reduced much and if you delay, someone else will take over that place. Hence, quality has to be ensured in a short span. These facts will drive anyone to be choosy about the devices when it comes to validation.

Let us have a look at the various devices that need to be considered for any mobile application testing.

Solution

By considering a couple of parameters, we can nail down on the devices on which app needs to be tested.

Parameters to be considered are –
1.    Type of devices
2.    Form Factors
3.    OS

Type of devices –
Now a days, most of the apps are made available in almost all types of devices and hence we need to ensure that the User Interface and User Experience Testing parameters for the app are met in all types of devices including phones, tablets and phablets.

Form Factor –
In most of the cases, the size of the screen is miss-understood with the resolution of the screen. These are the two exclusive parameters to be considered. Resolution is the number of pixels on the screen, irrespective of the screen size. How the app looks and objects placement on the screen are dependent on this parameter. There could be two devices of screen size 5 inch but their resolution might differ.

OS (iOS, Android, Windows and BlackBerry) –
The app is always built with the combination of new development and third party features or services, in the end product. There is a possibility where a developer would have tweaked third party features to meet the product’s requirement. Hence, we need to ensure that the combination of all these features works fine in multiple OS and their different versions that are majorly in use. There is no point in testing outdated versions of OS as the users will keep on moving towards the updated versions. But this has to though through as to which is the oldest version that we need to support for.

Matrix of devices shortlisted for testing
Note –

1.    Below information is as per Gsmarena.com.
2.    Only iOS and Android is considered in the below matrix to explain the exercise.

Type	Device	Resolution	OS versions	Size (inch)	Test Type
Android Tab	Nexus 10	2560 x 1600	5.X.X	10	Func & UX
Android Tab	Nexus 7	800 x 1280	5.X.X	7	UX
Android Tab	Micromax Canvas Tab P470	600 x 1024 pixels	4.4.x	7	Func & UX
Android Phones	Sony Xperia Z5	2160×3840	5.1.x	5.5	Func & UX
Android Phones	Nexus 6P	1440X2560	6.x	5.7	Func & UX
Android Phones	Samsung Galaxy S4	1080×1920	4.2.2	5	Func & UX
Android Phones	Moto G	720×1280	5.1.1	5	UX
Apple Phones	iPhone 5s	640 x 1136	iOS 7	4	Func & UX
Apple Phones	iPhone 5s	640 x 1136	iOS 9.x	4	Func
Apple Phones	iPhone 6	750 x 1334	iOS 8.x	4.7	Func & UX
Apple Phones	iPhone 6s	750 x 1334	iOS 9.x	4.7	UX
Apple Phones	iPhone 6s Plus	1080 x 1920	iOS 9.x	5.5	UX
Apple Tab	iPad Air 2	1536 x 2048	iOS 9.x	9.7	Func & UX
Apple Tab	iPad mini 2	1536 x 2048	iOS 8.x	7.9	Func

Though the above list is handpicked list of devices, it looks exhaustive and very difficult to test in all of them. The idea here is to cover all the form factors, OS and types of devices with different brands and hence the list seems to be big.

There is no shortcut if we have to validate the functionality in different OS versions and User Interface & User Experience factors in different form factors. Hence, the combination of devices and OS selection are done keeping these facts in mind. For different versions of OS, functionalities are validated to ensure that the newly developed piece of code and third party features are working fine without any functional flaws. For different Form Factors, UX parameters are validated to make sure that all the object in the screen are fitting properly  inside the screen as per the decided mock ups and there are no overlaps or partially hidden objects.

While we do functional validation on different devices it is obvious that you will make out the UI and UX glitches. So when you are testing only User Experience related scenarios you would know what is covered along with the functional testing and more focus has to be shown in the other areas.

One should always keep an eye on the market to know about the new devices or versions of OS or browsers that come to market and see if they fit into the above table. With this exercise, it is easy to arrive at the devices to be considered for testing.

Despite the short development cycles, go to market pressures and increasing competition in the mobility arena, it is key to do the mobile application testing across multiple devices and platforms and it is daunting too. Effective and timely mobile testing can enable device makers and application developers in collecting appropriate metrics that improve the overall quality of products and will be able to deliver amazing customer experiences.