Android Application Testing

Now with more than 60% market share, Android has many testing challenges due to an exponentially expanding testing matrix of various devices and OS versions. Android developers must not only ensure that each new app and version works on multiple device models from Samsung and LG to HTC and others, but they also have to worry about multiple OS versions and screen resolutions.

Some challenges particular to Android application testing are:

  • Device Fragmentation – There are more than thirty major mobile device manufacturing companies which use custom ROM. These devices cover a variety of screen size, pixel resolution, and Android Versions. It is not viable to fully test across the complete range of devices from different manufacturers with different specifications. The challenge is determining a “matrix” of devices that properly represent and over the target market.
  • Frequent OS Release – For Android devices, there are more than 10 versions of operating systems. Versions are constantly enhanced and updated to fix defects and to provide better user experience and performance. The challenge for the tester is ascertaining the best representative matrix of device OS versions to ensure proper
    coverage.
  • Various Networks Speeds – Even under the same network, but on different Android devices, the speeds of the network can vary. Mobile devices have a base bound, and they are not always the same, even within the same model but maybe produced.

Mobile Apps

XBOSoft’s specialized Android mobile app testing services can help you address your challenges. Android Application Automation Testing, Stress Testing Android Applications, and Android Application Penetration Testing are all important areas to consider.

Now with more than 60% market share, Android has many testing challenges due to an exponentially expanding testing matrix of various devices and OS versions. Android developers must not only ensure that each new app and version works on multiple device models from Samsung and LG to HTC and others, but they also have to worry about multiple OS versions and screen resolutions.

Some challenges particular to Android application testing are:

  • Device Fragmentation – There are more than thirty major mobile device manufacturing companies which use custom ROM. These devices cover a variety of screen size, pixel resolution, and Android Versions. It is not viable to fully test across the complete range of devices from different manufacturers with different specifications. The challenge is determining a “matrix” of devices that properly represent and over the target market.
  • Frequent OS Release – For Android devices, there are more than 10 versions of operating systems. Versions are constantly enhanced and updated to fix defects and to provide better user experience and performance. The challenge for the tester is ascertaining the best representative matrix of device OS versions to ensure proper
    coverage.
  • Various Networks Speeds – Even under the same network, but on different Android devices, the speeds of the network can vary. Mobile devices have a base bound, and they are not always the same, even within the same model but maybe produced.

Mobile Apps

XBOSoft’s specialized Android mobile app testing services can help you address your challenges. Android Application Automation Testing, Stress Testing Android Applications, and Android Application Penetration Testing are all important areas to consider.

Mobile Automated Testing

Organizations intending to deploy mobile applications must plan their testing strategy across manual and automated testing approaches using automation where and when it’s appropriate. As with regular automated testing, mobile app testing follows the same rules; it makes no sense to automate where it’s not economical. In other words, don’t automate tests that aren’t run often, and don’t automate tests what will need to be changed frequently due to an application that is changing often.

Mobile test automation is now commonly used to tackle platform and application complexity issues and increase coverage as the number of platforms continues to expand. Learn when and when not to consider using automation for mobile testing by reading this white paper (Mobile Test Automation with Robotium – Getting Started).

There are also many automation tools available to test mobile applications. Each has its own strengths and limitations and you may need a combination of several to do what you need. We’ve listed a few below but this is not exhaustive as new tools and versions are released frequently:

  1. Experitest – Accelerate mobile testing cycles and increase the quality of your releases with high-volume automated android testing.
  2. Robotium – Robotium is a free Android UI testing tool. It is suitable for test automation on different Android versions and sub-versions. Software developers often describe it as Selenium for Android. Tests created by Robotium are written in Java. In fact, Robotium is a library for unit tests, but it takes much time and effort to create tests by means of Robotium, as one must work with the program source code in order to automate tests. The tool is also unsuitable for interaction with system software; it cannot lock and unlock a smartphone or a tablet. There is no Record and Play function in Robotium, and it does not provide screenshots.
  3. MonkeyRunner – MonkeyRunner is one of popular Android Testing tools used for automating of functional testing. One does not have to deal with the source code in order to automate tests. The tests are written in Python and one may use a recording tool for creating tests. MonkeyRunner can run tests on real devices connected to a PC or emulators and has an API what allows it to control a smartphone, a tablet or an emulator from outside the Android code. A significant disadvantage is that it is necessary to write scripts for each device. Another problem of MonkeyRunner is that the tests require adjustments each time the user interface is updated.
  4. Ranorex – This is a licensed tool good for test automation of not only the latest, but beginning from Android 2.2. One of the Ranorex advantages is its detailed reporting with screenshots. It can connect a smartphone or a tablet to the Internet via Wi-Fi.
  5. Appium – Is a framework for creating automated tests for iOS and Android. It is a free tool. It supports Android versions from 2.3 and later. Appium utilizes WebDriver interface for testing and supports many programming languages, such as Java, C#, Ruby and others which are in the WebDriver library. It can manage Safari and Chrome on mobile devices, but some test engineers complain that it provides poor reporting.
  6. UI Automator – This tool supports Android versions beginning from 4.1, which represents about 66% of all devices using the Android operating system. UI Automator is able to interact with all types of Android products, including system applications. This enables UI Automator to lock and unlock a smartphone or a tablet. It can be integrated with testing framework TestNG to generate informative and detailed reports.

Lastly, in our White Paper, Mobile Testing Challenges and Solutions, learn more about how to handle:

  • Platform and sensor proliferation and fragmentation, perspectives into location, photography, and communications.
  • Application design considerations and how they impact testing requirements.
  • Actionable recommendations on how to best employ and achieve efficient software tests – from manual test approaches through large-scale automated test suites employing continuous integration.

Performance and Stress Testing Mobile Applications

Performance testing is much more complicated with mobile applications as the pieces are disjointed and not well integrated. There are three main components; server, network, and mobile device with application. You need to be cognizant of each component while determining your strategy based on browser-based testing versus native applications as discussed above regarding different types of mobile applications. In addition, you should take into account mobile application behavior (on device) under stressed resource conditions (memory, network) with various permutations of stress on the network and server components. Real multi-tasking is limited and mobile applications are prone to interruptions, such as incoming calls, SMS, or switching to other applications. Mobile users have a high expectation for the application performance and performance is a large component of user satisfaction and user experience. Therefore, performance testing is an essential component of mobile app testing when releasing an application.

Some Performance testing recommendations include:

  • Test in a production environment: Lab testing should be done first, but as with web applications, the production environment can behave differently.
  • Anticipated peak load: It is difficult to estimate peak load accurately. Note all your parameters and assumptions and do a sensitivity analysis to determine ‘what if’ this happened. Then, base your peak load on the worse case.
  • Test scenario profiles: Make sure that your performance tests accurately model the actual behavior and environment of your mobile users. They will never be logging in all at the same time, nor will they all be doing a search. Temporal considerations such as day of week and time of day will also make a big difference as users may behave differently (do different things on different days).
  • Test mobile applications across geographies: Performance will vary from different locations or countries, with varying levels of overhead and latency in different carrier networks.

For additional information on Android mobile app testing, please read our White Paper, Mobile Performance Testing – Techniques and Tips for Local Performance, to:

  • Understand performance aspects related to platform and app complexity issues.
  • Identify which platforms can and cannot be supported with acceptable performance.
  • Learn how to determine if it’s the CPU, memory, or another cause that’s slowing your app down.
  • See examples using commonly available tools.

Mobile Application Security

Most mobile apps communicate to a backend service, and those services are prone to the same types of attacks we are familiar with in web apps on desktop machines. In fact, the most dangerous vulnerabilities usually reside in the mobile back end (i.e. Web Services and APIs) and not in the application.

Although mobile apps differ in that there is inherently more security against injection and similar such attacks, mobile security instead, focuses around data protection on the device and the network. Some important areas that could be prone to security issues are:

  • Local Data Storage
  • Communication with Trusted Endpoints
  • Authentication and Authorization
  • Interaction with the Mobile Platform
  • Code Quality and Exploit Mitigation

The Mobile AppSec Verification Standard (MASVS)

The MASVS defines a mobile app security model and lists generic security requirements for mobile apps. It can be used by architects, developers, testers, security professionals, and consumers to define and understand the qualities of a secure mobile app, click here).

There many mobile app security testing tools that can be used to detect a wide spectrum of the most common security weaknesses and vulnerabilities, including OWASP Mobile Top 10. and provides a user-friendly report with the discovered issues. One good FREE tool can be found at ImmuniWeb. It can be used for:

  • Static Application Security Testing (SAST)
  • Dynamic Application Security Testing (DAST)
  • Behavior Testing for malicious functionality and privacy
  • Software Composition Analysis
  • Mobile Application Outgoing Traffic
  • Mobile App External Communications

Next Steps: Contact us for more information about our Android mobile application testing services.

What our clients have to say…

“They automated the web application testing by almost 95% within three months.”

- Lead QA, Whil

Whil. Logo

“They’re technically skilled and great at software development.”

- Director of PMO, MatrixCare

MatrixCare Logo

“They're very proactive and always follow through on tasks.”

- President, REAL Software Systems

REAL Software Systems Logo

"XBOSoft is able to see things from a different perspective and ask smart questions."

- CEO, Mobile MedSoft

Mobile Medsoft small

"Their work is very good and their attention to detail is excellent."

- Director of Engineering, Sago Mini

Sago Mini

"Our employees are always either requesting to work with
them or requesting to have more of their resources."


- Software Engineer, BlackLine

XBOSoft BlackLine case study

"XBOSoft has some of the most detailed and careful
test execution and reporting I’ve ever seen."


- DevOps Director, Bongo

Bongo White Logo

"They work hard and make the customer happy."

- Agile Coach, AKVA group Software AS

Akva Logo

“A CEO who’ll make an effort to visit you once a year to make sure you are happy with their
services is at the top of my personal list [of recommendations.]”


- Peter Menhart, Senior Lead Software Developer, Mitel

Akva Logo

“The key to any software application is to have the interface offer the flexibility that will drive
fast and accurate results. XBOSoft enables us to achieve these broad-based goals.”


- Project Manager, Adra Match

Adra Match Logo

“What I have seen so far is that you have done a great job!”

- Project Manager, AdminControl

AdminControl Logo

“In our early discussions and pilot testing program with XBOSoft, they demonstrated
impressive knowledge and experience in our type of CPM application. ”


- VP of Development, Host Analytics

Host Analytics Logo

"We have appreciated the high quality work your team
has done as well as your leadership"


- Board Member, Rightsline

Rightsline Logo

“"You've produced a good piece of work! Excellent!" ”

- Chief of Development, QuestBack

QuestBack Logo

“When our requirements change, they’re willing
to shift and adapt to what we need.”


- Software Developer, Telecommunications Company

Put our team of experts to work for you!

Put our team of experts to work for you!