Our testing team is regularly involved in quality assurance projects, so we’re constantly looking for new ways to optimize and improve testing processes. In addition to on-premise software testing, our team also conducts cloud-based testing, which allows us to significantly reduce time spent on testing and broaden the range of available devices and platforms. In this article, we share our experience using the BrowserStack and VirusTotal cloud-based testing tools. We analyze the benefits and pitfalls of these two cloud-based services. This article will be helpful for software testers who are looking for new application testing tools.
Cross browser and configuration testing are two major types of testing for web and mobile applications. Our testing team often runs into issues with not having a sufficient real-world environment for full-scale cross browser testing and not having enough real mobile devices for configuration testing of mobile applications. To overcome these challenges, we began to research cloud-based testing services. We looked for a cloud service (or several) that would allow us to effectively conduct cross browser testing as well as configuration testing on real mobile devices with the required test coverage. While investigating possible solutions, we found BrowserStack to be a single service that meets all our testing environment needs. It also turned out to be quite convenient to use, as you can test BrowserStack capabilities with its free trial version.
Google Chrome is the recommended browser for accessing the service. The company offers four services for testers. All you have to do is register online at the BrowserStack official website and pick the service you need. Here's a brief BrowserStack review.
Web testing services:
- Live – manual testing
- Automate – Selenium based BrowserStack automate feature
Mobile application testing services:
- App Live – manual mobile testing
- App Automate – automated testing based on Appium, Espresso, XCUITest, or EarlGrey
To test a web application, simply go to the Live section, select a browser, wait until it loads, and start working. To test a mobile application, go to the App Live section and choose one of the following options:
- Upload an installation file in APK or IPA format (for Android or iOS respectively)
- Upload your app from the Google Play Store (for Android apps)
- Upload your app from the App Store or TestFlight (for iOS apps)
There’s also an important and useful feature called Local Testing. For security purposes, our servers were working only within the local network, so we needed to use BrowserStack local testing option. And this functionality was available. You can find out more about Local Testing here.
We discovered the following pros of BrowserStack:
- An extensive selection of popular browsers with the ability to use them on different operating systems
- A large variety of mobile devices with various operating systems: Android 4.3–8 and iOS 8–11. This is a significantly wider choice compared to other testing services.
- Built-in automation services
- Option to change geolocation for mobile applications
- Technical support responds without delay
However, we also found some minor BrowserStack disadvantages to watch out for:
- Processing speed is slower than on actual devices
- Mobile device settings are unavailable
- The behavior of the device while receiving push notifications isn’t always consistent, so the notifications aren’t always received
- The behavior of the device while receiving geolocation data doesn’t always match reality (the device isn’t always able to determine its geolocation)
Fortunately, the service is actively developing and is in touch with its users. Just recently, the service’s tech support team asked for our feedback, so we had a chance to suggest a few improvements.
While testing one of our desktop applications, we needed to check for compatibility with various antivirus programs. We did some research and picked several of the most commonly used antivirus solutions for testing. Then we spent around three work days installing the necessary environment. After a while, we returned to this task when getting ready for the next release. Only this time we tried to find a more efficient way of checking for antivirus software compatibility.
One of our team members found VirusTotal, which allowed us to cut the testing time to only a few minutes. Currently, this service offers a simple way to test the compatibility of both Windows and macOS desktop applications (*.exe and *.dmg files) with over 50 antivirus programs. There’s also an option to test web applications.
VirusTotal doesn’t require configuration, as you only need to select a file on the hard drive to analyse it and get results.
This service isn’t without limitations, however. We ran into a few false positives while testing some of our Windows applications, meaning the actual antivirus program reported malware while VirusTotal did not.
Because of that, we still manually test for compatibility with antivirus software, but we do it less frequently.
To sum up, we’ve identified the following pros and cons of VirusTotal:
- It’s fast, reducing time spent on compliance testing by several times.
- It’s free, so there’s no need to purchase antivirus software licenses.
- Testing covers all major antivirus programs.
- It tests both macOS and Windows installation files.
- Sometimes there are false positives, periodically prompting us to validate results with actual behavior.
- File size is limited to 256 MB.
In this article, we shared our experience using the BrowserStack and VirusTotal cloud-based testing tools. We showed how to use these tools to test various applications and discussed their pros and cons. These tools allow you to overcome the scarcity of real testing environments and broaden your range of tested devices and applications. The drawbacks of these services can be compensated for by manual testing. Our Apriorit team has also tried to be in touch with the developers of these services to offer suggestions for improvement.
The Apriorit testing team has extensive experience testing desktop, web, and mobile applications. Contact us if your project requires testing expertise.