Mobile Testing: Why test and how to decide what to test?

Have you been assigned the task to test your website / app on ‘all’ mobile devices (i.e. phones and tablets), but you don’t know where to start? Then this and the following two posts will be perfect for you!

Why would you want to test on mobile devices?

It has been estimated that there are nearly 7 billion mobile subscriptions worldwide with more than half in Asia Pacific. Smartphone sales are showing strong growth worldwide and the mobile market is changing rapidly. Mobile devices are being used for financial transactions such as mobile banking and mobile money. Unlimited data plans are becoming very popular with extensive media sharing. Instant messaging is preferred over phone calls or meeting in person. Most of the applications on mobiles contain strong advertising options and companies are growing solely on mobile based sales.

Mobiles have given users more new features that weren’t possible before, including voice features (Siri, OK Glass) and location features (finding friends, tagging photos by location) etc. Mobile devices have become an extensive part of everyday life with more and more websites becoming responsive and more apps being launched than ever. Phrases such as ‘there’s an app for that’ are now true rather than the ‘joke’ they started out as: it is more than likely there really is ‘an app or that’.

With such high usage, it is apparent that your website / app must work perfectly on these mobile devices and therefore thorough testing is necessary. But testing websites and apps on mobile devices has been extremely hard, as there are no clear guidelines and standards. Although you can gain certification (for example, Mobile Testing Foundation Level or Certified Mobile Application Tester) it still doesn’t help you to start off testing easily.

Deciding which web browsers to test on is quite straightforward as there are relatively few to choose from. Many companies set a policy that testing should only be completed on web browsers, for example that have more than 5% of the market share. However, the mobile device market is vast by comparison, with so many makes and models available it is really difficult to decide exactly which devices, operating systems and mobile browsers to test on. It’s not feasible to test them all, so which ones will you focus on?

Conduct user needs analysis

One of the best ways to decide which mobile devices to test on is to conduct extensive analysis of existing and potential users. Find out which devices they are using and make sure your website / app is optimised for them. Keep your existing users happy!!

Look at your Analytics

If you’re redeveloping an existing site or app and the previous one had analytics tracking, make sure you take a look at the data that gives you details about your users and their behaviour. Google Analytics is a great tool for this, providing lots of in depth but informative and usable information.

Review mobile device statistics

You could (and should) then look at overall mobile statistics (this could be global or country specific, depending on the target market of your website or app), site analytics and your company policy (if there is one).

I have found the following statistic sites helpful for providing insights into mobile device usage:

Having done the above, you are then able to make an informed choice about where to focus your mobile device testing. If pushed to give a list of what I usually test on then it would look something like: devices and os for testing

Now you have your list of devices you want to test on, my next post will look at how to actually go about testing… see you there!

Related blog posts: