Our company has been testing mobile apps for quite some time now and I’ve just taken on a new role as a test lead on one of our projects where we primarily test mobile. I’m responsible for guiding our team members, so I thought I’d write a list of mobile app testing mistakes to avoid so the team can use it as a reference (and I can sleep better at night knowing they’re up to speed!). Here’s the list I’ve shared with the team.
Common Mobile App Testing Mistakes to Avoid
1. Placing more importance on UI than on features
Testing UI (user interface) is important because it gives the first impression to users who have the app, but you shouldn’t only focus on testing UI. If your app doesn’t deliver the features your users expect, the UI most likely won’t make sense. Make sure that you have clear paths or user scenarios identified for your different user profiles. Then test those features associated with those scenarios. I like to make sure that the users can get their job done (task or goal) with ease and with accuracy (avoiding mistakes). Sometimes the UI has influence on this, but many times it’s just a distraction.
2. Testing only on a single device
As we all know, there are many different devices on the mobile market. Some of them use the same platform, but some problems only occur on specific devices. Therefore, it’s important to test on the various devices and OS versions that will be used by your users. You can’t test all devices, so you’ll need to profile your user base as much as you can. Most of the time, when you do an analysis of who your users are, you’ll discover that you can cover over 90% of your users with a handful of devices (fewer than 10).
3. Being unfamiliar with the device/platform
No matter which device or platform you’re testing on, testers must understand each of those devices and platforms. The app is exclusively designed for each and the app’s interactions may vary between devices and platforms. If you don’t know them, you cannot test the app against the requirements and for that particular device. For instance, swiping left and right or up and down may have different behavior by design on different platforms, so you’ll need to understand this and many other platform-specific features thoroughly.
4. Only testing on emulators
Obviously, testing on emulators is less expensive, but real devices are more reliable and more effective for finding defects. Even if you decide to use emulators as your main testing devices, you need a few real devices to confirm your test results. I like to use emulators for basic smoke testing, and then real devices for deep testing on user scenarios plus focused exploratory testing.
5. No installation/upgrade testing
Installation and upgrade testing on an app is crucial. The installation is the first interaction a customer will have with your product, so any failures here will send them to your competitors. Make sure you not only test both initial installation and upgrade, but also delete the app and reinstall it to see what garbage is left (either purposefully or not) which may inhibit or help the user from reinstalling.
6. Not testing on networks
Customers generally use mobile apps while on the go, moving around within a mobile network or switching to wireless networks, so it’s important that your app can work properly across different networks and in transition. Also make sure you test in transition between online, offline and different frequency bandwidths.
The list is certainly not exhaustive, but it’s a start in helping me sleep better knowing or hoping that we’ve at least avoided these basic mistakes. We also regularly reference our mobile testing strategy to see if we’re going in the right direction and getting the most bang for the buck in our testing efforts. You can download it here.