Subscribe to receive all latest blog updates

At the moment, there is a great variety of competing browsers. In this situation, there are no common standards for displaying sites: each one of them may display a site in its own way, for example, by distorting the page content. Due to this fact, the “cross-browser” term emerged, that is the feature of a site to be displayed identically in all browsers.

Although the cross-browser issue emerged rather recently, there already exist a great number of tools for testing it.

All cross-browser testing tools vary in the number of supported browsers, ease to use, price, and also testing speed. These factors greatly complicate the choice of the suitable tool.

This article is devoted to the problem of choosing the right cross-browser testing tool. It lists the popular tools, their advantages and disadvantages.

This article would be useful for testers, who haven’t performed this kind of testing yet, or for people, who face the problem of choosing the suitable tool.

Written by: Anna Gorbatyuk,
Tester of Network Testing Team

1. Cross-browser testing

1.1. What is cross-browser testing

1.2. Cases to use the cross-browser testing tools

2. Cross-browser testing tools

2.1. IETester

2.2. BrowserShots

2.3. The Spoon

2.4. CrossBrowserTesting

2.5. NetRenderer

2.6. Browserling

2.7. BrowserCam

2.8. Adobe BrowserLab

3. The right way to choose cross-browser testing tool. Practical examples

Conclusion

Cross-browser testing

What is cross-browser testing

As we mentioned above, the content of a site may appear twisted in some browsers as the result of the differences in data interpreting. If a site is displayed incorrectly, a typical user leaves it. That is why you should perform cross-browser testing in order to ensure that the site is displayed correctly regardless of the browser, which it is opened with.

Cross-browser is a feature of a website to be displayed and work identically in all of the popular browsers. Identity means the absence of layout breakups and the same readability of represented information. The “cross-browser” term is often confused with the pixel by pixel correspondence, which is, in fact, another term.

Cases to use the cross-browser testing tools

It should be mentioned that any tools (including the cross-browser testing ones) may contain their own bugs. The ideal option is to have five most common browsers available (especially when they are installed on a clear machine). But not every tester has such possibilities. In most cases, it is easier and faster to use a cross-browser testing tool.

Despite possible bugs, the cross-browser testing tools allow you to save much time. As usually it is not possible to install different versions of the same browser on one machine, it will be necessary to reinstall a browser at least 3 times to check all versions. In most cases, it takes more time than the installation of 1 or 2 cross-browser testing tools.

I.e. the cross-browser testing tools should be used when you need to test your site on several browsers and on different versions of these browsers at the same time. But if the testing requires you only to open a site in 2 browsers of the latest versions, there is no need to use the tools, as it would take even more time.

Another advantage of using the tools is the possibility to test a site in different screen resolutions. Nowadays, this issue concerns not only websites, as more and more people use notebooks, tablet PCs, mobile phones, and other devices.

Cross-browser testing tools

2.1. IETester (http://my-debugbar.com/)

IETester is a good tool for testing Internet Explorer versions. It is easy to use and allows you to test the IE of 6, 7, 8, 9 versions. This is a desktop application (unlike the majority of other tools, which are online services).

IETester is only available for Windows. It supports 6 versions of IE.

In general, we can point out such advantages of IETester:

  • Free;
  • Suitable for quick testing;
  • Interactive (gives the possibility for functional testing).

The disadvantages of this tool are:

  • Supports only one browser, Internet Explorer;
  • Has some bugs in work with IE 9.0.

Pic. 1. IETester interface

2.2. BrowserShots (http://browsershots.org/)

BrowserShots gives the possibility to test a site not only in five most popular browsers, but also in less known browsers, such as Netscape, Avant, SeaMonkey.

As BrowserShots performs the cross-browser testing on virtual machines, there is a possibility to set such parameters as OS, screen resolution, color depth, support of JavaScript and Flash. Unfortunately, besides unpleasant insignificant defects in GUI part, there are problems in implementation of the basic functions of BrowserShots: some browsers don’t work, some of them can be tested only after half an hour of waiting.

BrowserShots is available for Windows, Mac OS and Linux. It supports over 60 combinations of browsers and OS’s.

Advantages of BrowserShots:

  • Wide range of different browsers and their versions;
  • The possibility to choose OS and screen resolution;
  • The possibility of free usage.

Disadvantages of BrowserShots:

  • Some of the declared browsers are not available;
  • Not suitable for quick testing;
  • Not interactive: cannot be used for functional testing.

Pic. 2. Browsershots interface

2.3. The Spoon (http://spoon.net/)

The Spoon allows you to work with different applications without installing them. These applications include 4 popular browsers: Safari, Google Chrome, Mozilla Firefox, and Opera. In order to work with The Spoon you need to install a plug-in.

The Spoon is currently available only for Windows. It supports 35 browser versions.

Advantages of The Spoon:

  • It’s free;
  • Supports the most common browsers and their versions;
  • It’s interactive (can be used for functional testing);
  • Supports Firefox and Opera mobile browsers.

Disadvantages of The Spoon:

  • Less widespread browsers are not supported;
  • Doesn’t work on Macintosh and Linux.

Pic. 3. The Spoon interface

2.4. CrossBrowserTesting (http://crossbrowsertesting.com/)

CrossBrowserTesting is one of the most popular cross-browser testing tools. Its popularity is stipulated by pretty simple interface and, that is more importantly, by a wide range of browsers, including the ones for mobile platforms.

CrossBrowserTesting is available for Windows, Mac OS, Ubuntu, and mobile platforms. It supports over 100 combinations of different browsers and OS’s.

Advantages of CrossBrowserTesting:

  • Wide range of different browsers and their versions;
  • Suitable for quick testing;
  • The possibility to choose OS and screen resolution;
  • Supports mobile browsers.

Disadvantages of CrossBrowserTesting:

  • Commercial (the price range is $29.95 to $199.95).

Pic. 4. CrossBrowserTesting interface

2.5. Netrenderer (http://netrenderer.com/)

Netrenderer is intended only for online testing with Internet Explorer. It is ideal for pixel by pixel testing. Its range includes 5 IE versions: 5.5, 6.0, 7.0, 8.0, and 9.0.

Advantages of Netrenderer:

  • Free;
  • The possibility of pixel by pixel site comparison in different versions of IE;
  • Suitable for quick testing.

Disadvantages of Netrenderer:

  • Supports only one browser, Internet Explorer;
  • Cannot be used for functional testing.

Pic. 5. Netrenderer interface

2.6. Browserling (http://browserling.com/)

Browserling is a relatively new online tool, but it has already gained certain popularity. It is easy to use and allows you to quickly test five browsers.

Browserling supports 48 browser versions.

Advantages of Browserling:

  • The possibility of free usage;
  • Over 40 versions of popular browsers;
  • Interactive;
  • The possibility of pixel by pixel comparison;
  • Suitable for quick testing.

Disadvantages of Browserling:

  • Doesn’t support mobile browsers;
  • Doesn’t support rare browsers.

Pic. 6. Browserling interface

2.7. BrowserCam (http://browsercam.com/)

BrowserCam has rich functionality: it allows you to test a site in over 90 browsers and perform functional tests. In addition, you can choose OS, which a browser should be launched under. There is also the Device Capture function that allows you to test mobile browsers.

Browsercam is available for Windows and Mac OS X. It supports over 90 combinations of browsers and OS’s.

Advantages of BrowserCam:

  • Wide range of different browsers and their versions;
  • The possibility to choose OS;
  • Supports mobile browsers.

Disadvantages BrowserCam:

  • Commercial (the price range is $19.95 to $89.95).

Pic. 7. BrowserCam interface

2.8. Adobe BrowserLab (http://browserlab.adobe.com/)

Adobe BrowserLab is a free tool with the possibility of interactive cross-browser testing. It is good for quick testing thanks to easy-to-use interface. It has few versions of browsers available, that’s why it cannot be used for full testing.

BrowserLab is available for Windows and Mac OS X. It supports 13 combinations of browsers and OS’s.

Advantages of Adobe BrowserLab:

  • Free;
  • The possibility to choose OS for testing;
  • Interactive;
  • The possibility of pixel by pixel site comparison;
  • Suitable for quick testing.

Disadvantages of Adobe BrowserLab:

  • Little choice of browser versions (only 13);
  • Doesn’t support mobile browsers;
  • Doesn’t support rare browsers.

Pic. 8. Adobe BrowserLab interface

The right way to choose a cross-browser testing tool. Practical examples

The choice of the tool greatly depends on the site specifics. For example, if your site target audience is US citizens, you should take into account that the majority of Americans use Internet Explorer. Another example: a site is intended to be used by Mac OS users. In this case the site should be thoroughly tested in Safari browser. To determine the specifics, it is preferable to speak to the customer and specify the target audience. Depending on the statistic data on end users, you may vary the number of browser versions. Thus you can significantly reduce the time spent on testing.

If there is no possibility to determine the preferences of end users, it is recommended to perform testing in at least 3-4 versions of each of the most popular browsers, which are Internet Explorer, Mozilla Firefox, Google Chrome, Opera, and Safari (according to the data for the beginning of 2012).

If the customer has no statistic data available, you may use the data from this website: http://gs.statcounter.com/. This resource provides the statistics on browser usage in a country/region for a certain period of time.

Pic. 9. The browser popularity rating on the beginning of 2012 (worldwide results)

Let’s move on to the examples of the tool choosing.

  • Internet Explorer testing

Internet Explorer has remained a popular browser for years. Despite the fact that IE has lost positions with the release of Google Chrome, it is still holds the first position in usage worldwide (according to the data from gs.statcounter.com site). Due to this fact, it is very important to test a site in basic IE versions.

There are two popular tools for testing Internet Explorer: IETester and NetRenderer. There are a number of differences between these tools: firstly, IETester is a desktop application, unlike the Netrenderer internet resource; secondly, Netrenderer provides screenshots, while IETester allows you to test the performance of links, registration, system authorization. Netrenderer is better for pixel by pixel testing, that is why I would recommend IETester in this situation.

  • Quick test in basic browser versions

It would not be necessary to use the cross-browser testing tools for quick testing: it would be enough to perform tests on 5 most common browsers (if they are installed already), but it is better to target end users and select browsers for testing according to the statistics.

Let’s take the data on usage of different versions of browsers from http://gs.statcounter.com/ website. We get such results:  IE (6.0, 7.0, 8.0, 9.0 versions), Chrome (15.0 and 16.0 versions), Firefox (3.6, 8.0, 9.0 versions) and Safari (5.0 and 5.1 versions).

Pic. 10. The browser versions popularity rating on the beginning of 2012 (worldwide results)

There are 3 tools suitable for quick testing: Adobe BrowserLab, Browserling, and The Spoon. Adobe BrowserLabs doesn’t support Google Chrome of 15.0 and 16.0 versions and Mozilla Firefox of 3.6, 8.0, and 9.0 versions, so it is not suitable in this situation. Browserling supports greater number of versions, including Safari, that’s why it is better to choose Browserling for quick testing.

  • Full cross-browser testing, free tools

Browserling has the necessary qualities for full testing. Its biggest advantage is the wide selection of browser versions and interactivity that allows you to completely test the site content.

  • Full cross-browser testing, commercial tools

The most appealing by quality-price ratio is CrossBrowserTesting. It supports over hundred combinations of browsers and OS’s, allows you to choose the screen resolution on the selected OS, and to perform the testing of mobile browsers.

  • Mobile browser testing, free tools

Unfortunately, there are very few free tools for full testing on mobile browsers. The only suitable option is The Spoon.

  • Mobile browser testing,commercial tools

The best choice for testing in mobile browsers is BrowserCam. It is a little bit cheaper than CrossBrowserTesting and provides more variants of mobile browsers, including the old versions of IE, where, as a rule, the most problems are detected.

  • Rare browser testing, free tools

Only Browsershots has a wide choice of rare browsers among the free tools.

  • Rare browser testing, commercial tools

In this case, the most suitable tool is CrossBrowserTesting. It has a wider range of rare browsers.

Conclusion

The chapter 3 of the article describes some possible variants of the selection of the tool. While writing this article, I took into account typical problem situations, but you might not have made your choice yet. To ease the selection, I have created the following table of tools, where tools are compared according to the number of browsers and their versions, interactivity, price and testing speed.

#

Tool name

Price

Number of browsers

Rare browsers

Mobile browsers

Interactive

Testing speed

1

IETester

Free

6

No

No

Yes

Fast

2

Browsershots

Free

60+

Yes

No

No

Low

3

The Spoon

Free

35

No

Yes

Yes

Normal

4

CrossBrowserTesting

$29.95-$199.95

100+

Yes

Yes

Yes

Fast

5

NetRenderer

Free

5

No

No

No

Fast

6

Browserling

Free

48

No

No

Yes

Fast

7

BrowserCam

$19.95-$89.95

90+

Yes

Yes

Yes

Normal

8

Adobe BrowserLab

Free

13

No

No

Yes

Fast

Tab. 1. Tools comparison table

I hope the article will help you to choose the most suitable tool.

Good luck with testing!

 

Read another QA blog post about virtualization in software testing.

Learn about QA estimation techniques.

This article is created by our testing department - take a look at those written by Apriorit researchers, for example iOS reverse engineering post.