Ensuring the smooth and secure work of a mobile application requires quality assurance (QA) engineers to expertly pick which tests to perform for specific types of apps. Only then can they verify an app’s quality and keep up with the fast-paced cycle of mobile app development as well as meet end users’ expectations.
In this article, we discuss how to test a mobile application, which types of testing and checks to apply, and which device virtualization services to rely on. It will be useful for development companies that want to improve their mobile testing and enterprise mobile application management routines.
Why invest time in application testing?
Testing is an essential stage of any software development process that helps to ensure an application’s quality, usability, and security. However, testing sometimes gets overlooked in favor of allocating more time for app development or a faster release. Limiting or skipping testing activities is undeniably a bad development practice, but some organizations use it to save time or because they’re sure that their code doesn’t have major issues.
In fact, paying close attention to testing has many benefits and is definitely worth the time. Here is why mobile testing is important:
- Ensure a flawless user experience. Users expect your application to be easy to use, perform well, and work smoothly on their devices. If it doesn’t meet their expectations, users will use the app as seldom as possible or simply uninstall it.
- Protect user data from leaks and hacks. Some applications need to collect users’ personal, financial, medical, or other types of sensitive data to work properly. The organization that owns an application has to make sure that all sensitive data is well protected. A leak or theft of sensitive information can result in huge reputational and financial losses.
That being said, the testing stage may indeed take a long time, which is not always affordable for mobile applications with a short development cycle. In the next section, we discuss the types of mobile testing that must be performed to ensure your product has the right level of quality, performance, and security.
5 key types of mobile application testing
Mobile application quality assurance is a complex process composed of multiple reviews and checks. You can speed up testing by reviewing the scope of tests and running only those that are necessary. There are five key types of mobile application testing:
Besides these testing activities, each application may require additional checks depending on its goal and characteristics. For example, banking applications require many more security testing activities than gaming apps, yet the latter require additional performance and usability testing.
In this article, we focus on five main checks that can be applied to a wide range of mobile applications. Let’s start with the definition of functional testing and key checks.
Functional testing of mobile applications is the first and probably the most essential type of testing you should perform. Its goal is to ensure that the software successfully does what it’s supposed to do. Here’s a more detailed definition by the International Software Testing Qualifications Board (ISTQB):
To prepare functional test cases, you first need to gather the requirements that are usually defined in the app’s documentation. If there isn’t enough information, you can also define the needs of your target audience and turn them into requirements. Also, analyze the functions of competitors’ products and applications similar to yours and add relevant functionality to your list of requirements.
The requirements elicitation meaning in software engineering is that it allows you to compile a list of key functionalities your application should have. For example, if you’re testing an app for social media, its key functionalities will probably be user registration, creating and editing user profiles, messaging, and so on. This is the foundation of your app that you should test first.
Here’s a list of core mobile functional testing and QA activities:
Installing, updating, and deleting the application via an app store or using the APK or IPA file
Launching, closing, and minimizing the application
User interface (UI) elements performance
Switching between pages and UI tabs
Access to native device functionality
Disk space consumption
Using different types of internet connections
Receiving a call, text, or notification
Support for payment systems
This type of testing helps you ensure that your application is easy to use and understandable for your target audience as well as to find ways to improve the user interface (UI) and user experience (UX).
There are five criteria for assessing an application’s usability that we use in our mobile development projects:
As you can see, these are subjective criteria, which is why it can be tricky for QA specialists to test your application in a completely unbiased manner. Unbiased testing becomes challenging to organize when your application needs several testing iterations. A QA team can get used to unusual behavior and overlook a usability issue.
That’s why using alpha and beta testing with focus groups is particularly useful for checking an app’s usability. Gather a group of people (who can be the end users of your software) and prepare user scenarios that they need to perform. After the testing session, collect focus group feedback on the five usability criteria we discussed. This way, you can get an unbiased and insightful assessment of your application in general and its usability in particular.
Here’s the minimum mobile app usability testing checklist:
|Checked functionality|| |
Switching between pages and UI tabs
Support for touchscreen features
Users expect your application to work as fast as possible, or at least not to be irritatingly slow. Testing your app’s performance allows you to make sure the app works fine on different devices and under different loads. Performance testing of mobile applications includes checking the productivity of devices, servers, and networks.
Let’s take a look at the key checks you need to conduct.
1. Device performance
To ensure the smooth work of your application, test it using several popular devices with various types of processors, amounts of RAM, battery capaсities, etc. If you don’t have enough real devices to test on, you can use virtualization services that we overview later in this article.
Here are the key tests to run on your devices:
CPU and RAM load
2. Server performance
Server response times and data exchange can become a performance issue when a lot of users interact with your application. Check the following to ensure that communication with the server doesn’t slow down your application:
Data exchange with the server
|Request to server sent via the application’s API|
3. Network productivity
If your application requires an internet connection, you need to test how it works when connected to various types of networks. Here’s how you can do it:
Mobile device manufacturers produce many smartphones with unique operating system versions, hardware, screen sizes, resolutions, etc. Testing the compatibility of your application with all devices is impossible. That’s why you need to understand what pool of devices your target audience uses and focus your testing efforts on them.
There are two ways of defining the pool of devices you need to prioritize in compatibility testing:
- Research the most popular smartphone models worldwide or in your target region.
- Gather statistics on mobile devices your audience uses with Google Analytics, Mixpanel, Adobe Analytics, FoxMetrics, Fair Analytics, or similar services.
Here are the key compatibility parameters you need to test:
Even with a limited number of device models, these three parameters and their combinations take a lot of time to test. To speed up this process, you can use pairwise testing. This test design technique combines tested parameters into pairs and tests all possible combinations for these pairs. With pairwise testing, you’ll be able to quickly detect lots of compatibility issues. But there’s always a risk that minor issues will slip through your tests, so be attentive when applying this technique.
An application’s security is a specific area of testing performed by security specialists. It helps ensure that users’ data collected by the application is secure and that malicious actors can’t abuse the app and get a hold of that data.
It’s hard to define a universal list of security checks since they heavily depend on the nature of your application, its architecture, and the types of data it collects. The ISTQB Syllabus for Mobile Application Testing Foundation Level [PDF] suggests considering at least these tests:
- Inputs for code injection and overflow
- Encryption of transferred data
- Encryption of locally stored data
- Deletion of temporary data after use or after an abnormal app termination
- Clearing of text in password fields
Of course, this isn’t a definitive list of must-have security tests. When planning your own tests, make sure they cover these widespread mobile security risks:
Now it’s time to take a closer look at ways you can successfully ensure the smooth performance, high efficiency, and reliable security of your mobile application even when you don’t have enough physical devices to test it on. In the next section, let’s take a look at the key mobile virtualization services.
Mobile virtualization services: a brief overview
Using virtual devices brings many testing benefits including speed, scalability, and access to any type of testing environment. We examined the types, benefits, and drawbacks of virtualization in one of our previous articles, where we compared it with cloud computing and discussed key differences between the two technologies.
Despite offering many advantages, a virtual device can’t fully emulate the behavior of a real smartphone or tablet. That’s why you should run your test cases on at least one real device before switching to a virtual one. Testing on a real device helps you make sure that your app works as intended in real life, and virtual testing allows you to check more mobile hardware and software combinations.
You can add virtual devices to your testing environment in two ways:
- Create the required virtual machines yourself using official Android, iOS, and other manuals. Go with this option when you don’t need a lot of additional devices.
- Use a mobile virtualization service.
At Apriorit, we have experience working with different cloud mobile virtualization services in our mobile development projects. Here’s a brief overview of the strengths and weaknesses of the services we use the most:
AWS Device Farm is a paid cloud service that allows you to remotely test your app on physical devices that your users work with. This service helps you see how a user will interact with your app on devices with different processors, locations, firmware, and memory sizes. However, almost all devices provided by AWS Device Farm are Android-based. At the time of writing, this service included only one Apple device model — iPhone 13.
Corellium is a paid cloud platform that provides access to a large pool of virtual iOS and Android devices, including devices with root privileges. Virtual machines from this service are configured as close as possible to real devices, which allows testing to be as close as possible to the real user environment.
Genymotion is an easy-to-use multifunctional emulator that has both free and paid versions. It works with powerful OpenGL graphics cards, has Android Studio plugins, and is available on several Windows, Linux, and macOS operating systems. Genymotion also allows you to use the webcam for an emulated device as a camera and execute ADB commands using the ADB tunnel. The key drawback of this service is that it only emulates Android devices.
BlueStacks is a popular cloud platform that emulates Android devices. It has free and paid versions, and it works well with both Windows and macOS. Although it can emulate any Android application, BlueStacks is designed specifically to improve the gaming experience of Android video games on Windows.
BrowserStack is a paid cloud-based testing platform that provides access to over 3,000 real on-demand mobile devices and browsers. This service helps to test the compatibility of websites and mobile applications with browsers, operating systems, and real devices. However, BrowserStack doesn’t provide you with full control over device settings.
In case you’re also working on drivers, check out our article on testing a Windows driver.
In a fast-paced mobile app development cycle, testing teams have limited time to ensure an application’s quality and even to define test metrics in software testing. Experienced QA engineers can use this time in the best possible way because they know which tests they should run for different mobile app testing types, which tools and services to apply, and what software performance testing best practices to use.
In this article, we provided you with a brief overview of mobile testing types and must-have checks as well as reliable device virtualization services you can use. Based on this knowledge, you can roughly estimate how much time and effort you’ll need to test your app and find ways for improving the process.
Want to make sure your software is tested quickly and thoroughly? Reach out to our expert QA team to leverage their knowledge!