Logo
blank Skip to main content

Automated Testing System Deployment on the basis of Team Foundation Server and Microsoft Test Management

QA

This article is intended to provide full and detailed description of deployment of automated testing system consisting of Team Foundation Server + Microsoft Test Manager + System Center Virtual Machine Manager.

This article may be useful for Automated Testing Specialists and System Administrators.

Introduction

Here is the general scheme of testing:

im1

General System Scheme

There is a physical server managed by Windows Server 2008 R2 x64. There is a Hyper-V role set on it. There is also a virtual internal network created in Hyper-V manager, where all virtual machines depicted on the scheme are hosted. As this is an internal network, physical server also has virtual network interface that enters this subnetwork. Moreover, physical server is the domain controller of this virtual subnetwork. All virtual machines belong to this domain. To get access to the advanced virtual machine management features, we will later install SCVMM Server on physical server. To make it easier to configure our system, we turn off UAC and Firewall on physical server.

As on the scheme, we need such virtual machines:

  1. TFS Server. It is a virtual machine managed by Windows Server 2008 R2 x64. There will be Test Controller installed on it without any specific requirements, and TFS will be also installed, which has high requirements as for performance. Besides, we need significant disk space as we are going to store all autotests code.
  2. TFS Client. It is a virtual machine managed by Windows 7 x64. Visual Studio 2010 and Microsoft Test Manager are installed on it. Here we can develop and debug tests and also manage the whole testing system, start tests, browse their results.
  3. Test Agent. It is a virtual machine where tests will be performed. In particular in our system there 10 of them: from Windows XP up to Windows7 including server platforms and x64 systems. There will be a Test Agent, Lab Agent and Build Service installed on each of them, but basically requirements to virtual machine concern only OS and tested product requirements.

 

1. TFS Server Configuration

On TFS SERVER

  • Clean Windows Server 2008 R2 is installed
  • UAC and Firewall are turned off.

1.1 Installing SQL Server 2008 R2

Full correct installation is shown in the video: http://elhajj.wordpress.com/2011/03/02/how-to-install-sql-server-2008-for-tfs-2010/

If the link to the video is invalid do the following:

  1. Start SQL installation
  2. Select new installation or add features to an existing installation
  3. Setup Support Files page โ€“ Click Ok.
  4. Product Key page โ€“ Click Next
  5. License Terms โ€“ Select the I have read and accept the license terms and click Next
  6. Setup Support Files โ€“ click Install, then click Next
  7. Setup Role Page โ€“ select the SQL Server feature installation.
  8. Feature selection page:

Check only:

For TFS:

  • Database Engine Services
  • Full-Text Search

For reporting:

  • Analysis Services
  • Reporting Services

Not required for TFS but useful for connection and database checking:

  • Management Tools Basic
  • Management Tools Complete

DO NOT CHECK:

  • Client Tools Connectivity.

Click Next

  1. Installation Rules page โ€“ click Next
  2. Instance Configuration page โ€“ click Next
  3. Disk Space Requirements page โ€“ click Next
  4. Server Configuration page:
  5. Click Use the same account for all SQL Server services
  • Account name: NT AUTHORITYNETWORK SERVICE
  • Do not enter any password
  • Click Ok
  • For SQL Server Agent and SQL Server Browser Services set the Startup Type โ€“ Automatic.
  • Click Next
  1. Database Engine Configuration page:
  • Select Mixed Mode radio button
  • Set and confirm the Password for sa user
  • Click the Add current user button.
  • Click Next
  1. Analysis Services Configuration page
  • Click the Add Current User button
  • Click Next
  1. Reporting Services Configuration page
  • Remain the Install the native mode default configuration radio button selected.
  • Click Next
  1. Error Reporting page โ€“ click Next.
  2. Installation Configuration Rules page โ€“ click Next.
  3. Ready to Install page โ€“ click Install

Finish.

1.2 Install IIS.

  1. Open the Start -> Administrative Tools -> Server Manager
  2. Right Click the Roles -> Select the Add Roles item
  3. Before You Begin page โ€“ click Next
  4. Server Roles page – check the Web Server (IIS) and then click Next
  5. Web Server (IIS) page โ€“ click Next
  6. Role Services page – check All options and click Next
  7. Confirmation page โ€“ click Install

Close.

1.3 Install TFS Server.

The manual is supplied with the installation files. You can read about TFS installation there: TFSInstall.chm

Or you can download it here: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24337

Also there is useful video here: http://msdn.microsoft.com/en-us/vstudio/video/gg537701

Step-by step installation instruction:

  1. Start Setup.exe
  2. Welcome Screen โ€“ click Next
  3. Accept License Agreement โ€“ Select the I have read and accept the license terms and click Next
  4. Select Features to Install โ€“ check All options โ€“ click Install
im2

Features to install

  1. TFS asks for restart โ€“ click Restart Now.
  2. When installation is Finished click Configure.

1.4 Configuring TFS Server

*Optional: If you have closed TFS Setup wizard: Open Start -> All Programs -> Microsoft Team Foundation Server 2010 -> Team Foundation Administration Console.

  1. Click the Application Tier and then Click the Configure installed Features.
im3

Start configuring Application Tier

  1. Remain Standard Single Server selected and click Start Wizard
im4

Standard Single server Start Wizard

  1. Welcome page โ€“ click Next
  2. Service Account page:
  3. Specify Account name and password for access to MS SQL Server which installation was described above.
  4. Click Test
  5. If the test is successful click Next.
im5

Specify account

  1. Review page โ€“ click Next.
  2. Readiness Check page:
  • Firewall and SharePoint are not important for us now, don’t worry about the warning. Click Configure
  1. Configure page โ€“ click Next.
  2. After installation completes click Close.
  3. Application Tier configuring is finished.
  4. Select the Configure Team Foundation Server Proxy and click Start Wizard.
im6

Configuring proxy start wizard

  1. Welcome page โ€“ click Next
  2. Service Account page โ€“ click Next
im7

Service account

  1. Proxy Details page โ€“ click Next
im8

Proxy details

  1. Review page โ€“ click Next
  2. Readiness Check page โ€“ click Configure
  3. Configure page โ€“ click Next
  4. Complete page โ€“ click Close
  5. The Proxy configuration is finished
  6. Select the Configure Team Foundation Build Service and click Start Wizard.
im9

Start build service configuration

  1. Welcome page โ€“ click Next
  2. Project Collection page โ€“ click Next
  3. Build Services page โ€“ click Next
im10

Build Services

  1. Settings page โ€“ click Next
im11

Settings

  1. Review page โ€“ click Next
  2. Readiness Check page โ€“ click Configure
  3. Configure page โ€“ click Next
  4. Complete page โ€“ click Close
  5. Build Service Configuring is finished.
  6. Click Close.
  7. Now you may close the Team Foundation Server Administrative Console.

On the TFS Server:

1.5 Install Test Controller

  1. Run setup.exe
  2. Welcome page โ€“ click Next
  3. License Agreement page โ€“ select the I have read and accept the license terms and click Next
  4. Select Features to Install page โ€“ Remain the Microsoft Visual Studio Test Controller option checked and click Install.
im12

TC Select features to install

  1. Installation is finished. Click Configure.

1.6 Configure Test Controller

  1. * Optional: Click Start -> All Programs -> Microsoft Visual Studio 2010 -> Microsoft Visual Studio Test Controller 2010 Configuration Tool
  2. Remain Network Service Account selected
  3. Check the Register with Team Project Collection option
  4. Enter the TFS link. The link that was created previously is http://tfs:8080/tfs/defaultcollection
  5. We will not execute load tests, thatโ€™s why it remains unchecked. We can reconfigure Test Controller later if any.
  6. Click Apply Settings.
im13

TC Configuration Settings

  1. Close all windows of Test Controller Configuration Tool.

1.7 Install MS Visual Studio 2010 Ultimate

  1. Start setup.exe
  2. Click Install Microsoft Visual Studio 2010
  3. Welcome page โ€“ click Next
  4. License Agreement page โ€“ select the I have read and accept the license terms and click Next
  5. Select Features to Install page :
    • Select Custom
    • click Next
  6. Select Features to Install next page:

Check:

  • Microsoft Visual Studio 2010 Ultimate (and sub-nodes)

Uncheck:

  • Microsoft Office Developer Tools (x64)
  • Dotfuscator Software Services Community Eddition
  • Microsoft SQL Server 2008 Express Service Pack 1 (x64)
  • Microsoft Share Point Developer Tools.

Click Install

  1. When installation asks for restart click Restart Now
  2. After restart and log in the installation continues.
  3. When installation finishes click Finish.

1.8 Create Test Project

1.8.1 Create new team foundation project

  1. Open Start -> All Programs -> Microsoft Visual Studio 2010 -> Microsoft Visual Studio 2010
  2. Click Connect to Team Foundation Server
im14

Connect to tfs

  1. Click the Servers button.
im15

Connect to tfs server list

  1. Click the Addโ€ฆ button
im16

Add tfs server

  1. Enter the Team Foundation Server link: http://tfs:8080/tfs and click Ok.
im17

Enter TFS server link

  1. TFS server is added to the list. Click Close.
im18

TFS server is added

  1. TFS Server is added to the drop-down list. DefaultCollection is displayed in the list of Team Project Collection.
  2. Select the Default Collection and click Connect.
im19

TFS server is added to drop-down list

  1. Team Explorer viewer opens.
im20

Team Explorer

  1. Right Click the tfsDefaultCollection node
  2. Select the New Team Projectโ€ฆ menu item.
im21

New team project

  1. Specify the name of the Team Project and click Next.
im22

Team project name

  1. Select the MSF for Agile Software Development v5.0 process template and click Finish.
im23

Process template

  1. When Team Project creation process is finished, uncheck the Launch the process guidance for more information about running the team project and click Close.
  2. On Welcome page Click the New Projectโ€ฆ
im24

Create new project locally

  1. Create new project with such settings:
  • Select Visual C# -> Test
  • Check the Add to Source Control option
  • Remain all other settings default.
  • Click Ok.
im25

Create new project settings

  1. The Add Solution to Source Control form appears. Remain all settings default and click Ok.
  2. The Project is added to the Source Control.
im26

Project is added to source control

1.8.2 Record Coded UI test

  1. Letโ€™s create an UI test
  2. In the Solution Explorer, right Click the Project1 node and select Add -> Coded UI Testโ€ฆ
im27

Add coded ui test

  1. Remain the Record actions, edit UI map or add assertions selected. Click Ok.
im28

Generate coded UI test

  1. The recorder appears. Click the Start Recording button.
im29

Recorder

  1. Recording starts. The button icon was changed.
im30

Recording starts

  1. Click Start -> Computer
  2. Open C:Users
  3. Right click and select the New -> Folder in context menu.
  4. Enter name โ€œ1โ€
  5. Press Enter
  6. Click the Pause Recording button of the Test Recorder.
  7. Click the Generate Code button.
im31

Click Generate Code

  1. Remain the method name default and click the Add and Generate button.
im32

Test method Add and Generate

  1. Code is generated.
  2. Right click the Solution node and select Check Inโ€ฆ item in the context menu.
im33

Check In

  1. Enter any comment to the Comment text box and click the Check In button
  2. The items in the Solution Explorer viewer are displayed as Checked in.
im34

Checked in

1.8.3 Create a build definition and build the project

  1. Letโ€™s build the project.
  2. Open the Team Explorer viewer
  3. Right click the Builds node and select the New Build Definitionโ€ฆ item in the context menu.
im35

New builds definition

  1. On the General page, remain the default Build Definition name.
im36

Build definition

  1. Open the Build Defaults page.
  2. Remain Build Controller default.
  3. On the TFS Server machine, create a folder for Builds output and share it. For example C:ShareBuilds
im37

Shared folder for build output

  1. On to the computer with Visual Studio, enter the path to the build output to the text box: TFSShareBuilds
im38

Enter build output path

  1. Open the Process page
  2. Expand the Advanced node.
im39

Build process

  1. Set the Disable Tests value โ€“ True.
im40

Disable tests

  1. Save the project (click the save All button on VS toolbar).
  2. The build definition is saved.
im41

Build definition saved

  1. Letโ€™s build the test.
  2. Right click the Project1 build definition and select the Queue New Buildโ€ฆ context menu item.
im42

Queue new build item

  1. On the Queue Build โ€œTestProjectโ€ remain all settings default and click Queue.
  2. Build is started.
im43

Build is started

  1. The build is finished successfully.
im44

Build is finished successfully

On AGENT:

Before installing test agent:

ATTENTION!!! TURN OFF the IPv6 PROTOCOL on agent.  If IPv6 protocol is turned on, it is used by default, but Test Agent does not support IPv6 connection.

ATTENTION: If you install Test Agent on WinXPx64 or Server2003 x64/x86, you should additionally install

1.9 Test Agent installation

  1. Installing Visual Studio Agent 2010. Start setup.exe
  2. Welcome page โ€“ click Next
  3. License Agreement page โ€“ select the I have read and accept the license terms and click Next
  4. Select Features to Install โ€“ Remain the Microsoft Visual Studio Test Agent 2010 selected and click Install.
im45

Features to install

ATTENTION!!! For Win7 x64, Vista x64 and Server 2008 x64, select also the Performance Tool option. Otherwise the error will appear when configuring Test Agent.

See details here: http://social.msdn.microsoft.com/Forums/en-US/vststest/thread/b4f4951c-5faa-4553-bcc1-0c0ad3867e72

  1. When installation is finished, click Configure.

1.10 Test Agent configuration

  1. *Optional: Open Start -> All Programs -> Microsoft Visual Studio 2010 -> Microsoft Visual Studio 2010 Test Agent Configuration Tool.
  2. Select Run the test agent as Interactive Process. It is required for desktop application testing.  Click Next.
im46

Run agent as process

  1. On the next page, select such settings:
    • Enter the login and password of the current computer admin user to start agent as interactive process.
    • Check option Log on automatically.
    • Check option Ensure screen saver is disabled.
    • Check the Register with Test Controller option
    • In the text box, specify the test controller name and port tfs:6901 (this port is default for test controller). The test controller installation was described above.
    • Click Apply Settings.
im47

Configure test agent

  1. Configuration is finished successfully. Agent icon appears in the tray.
im48

Configuration is finished

TROUBLESHOOTING: If you get an error: โ€œFailed to open the Visual Studio v10.0 registry key. Ensure that test agent is installed, the config utility has the required permissions to access and modify the registryโ€ Do the following:

  1. Start setup.exe again
  2. Select the Add/remove features.
  3. Check the Performance Tool option.
  4. Complete the installation.

See details here: http://social.msdn.microsoft.com/Forums/en-US/vststest/thread/b4f4951c-5faa-4553-bcc1-0c0ad3867e72

  1. You may close all configuration forms.

1.11 Creating Test Plan, Test Cases and Test Environment

  1. On the computer where Visual Studio is installed, open Start -> All Programs -> Microsoft Visual Studio 2010 -> Microsoft Test Manager 2010
  2. Expand the tfs -> DefaultCollection and select the TestProject. Click Connect Now.
im49

TM Connect to TFS

  1. Test Manager requires Test plan. Letโ€™s create it. Click the Add button.
im50

Click add

  1. Enter test plan name, for example TestPlanForTestProject. Click Add.
im51

Enter test plan name

  1. Test Plan is created and selected. Click Select Plan.
im52

Test plan is selected

  1. TestPlan is opened in the Test Manager.
  2. Letโ€™s add a test case. Click the New button.
im53

Test manager opens

  1. Enter the test case name โ€“ โ€œCreate a folderโ€
  2. To add a step, click the cell in the Action column and enter steps, then add expected result. Click the Save and Close button.
im54

Test case is created

  1. Click the Testing Center and select Lab Center from drop-down list.
im55

Lab center

  1. On the Environments form, click the New button and select New Physical Environment from the drop-down list.
im56

Environments form

  1. On the Name and location page, enter the name of the environment โ€“ โ€œWin7โ€. Click Next.
  2. On the Machines page, select the agent name in the Available machines list and click Add to environment. The machine with test agent must be present in the list, if the test agent is online.
im57

Select test agent

  1. The agent was moved to the Selected machines list.
  2. To assign a role – click it and select the Desktop Client from the drop-down list.
im58

Select role

  1. Click Finish. The environment is created.
im59

Environment is created

  1. Return to the Testing Center (clicking Lab Center)
  2. Select the โ€œTestPlanForTestProjectโ€ test plan. Click Properties.
im60

Test case properties

  1. In Automated Runs, select the Win7 environment from the list.
im61

Assign test environment to test plan

  1. In the Builds -> Build in use click the Modify link.
im62

Assign test environment to test plan

  1. Click Save Plan and Select Build.
  2. The latest TFS build is selected in the Available builds combo box. Click the Assign to plan button.
im63

Assign build to plan

im64

Build is assigned

  1. Close the Assign Build form.
  2. Open the Microsoft Visual Studio.
  3. Open the Solution Explorer and build the project locally (Build -> Build Solution).
im65

Build solution

  1. Solution is built
  2. Letโ€™s associate test case with UI Test method. (you can find additional information here: http://msdn.microsoft.com/en-us/library/dd380741.aspx)
  3. Open the Team Explorer -> Test Project -> Work Items -> Team Queries -> My Test Cases. Double click My Test Cases, and the Create a folder test case appears in the viewer.
im66

Open test case

  1. Open the Associated Automation tab. And click the โ€ฆ button.
im67

Associated automation

  1. Select the CodedUITestMethod1 and click Ok.
im68

Methods

  1. Test method is assigned.
im69

Method is associated with test case

  1. Click the Save All button.
  2. Open Test Manager
  3. Click the Test item in main menu.
  4. Select the Create a folder test case and click Run.
im70

Click run

  1. Test Run form opens.
  2. Click the View link in the Test run log.
im71

Test run

  1. Log appears. The last record says that test run is completed.
  2. Close test run log.
im72

Test run log

  1. Click the Stop run button.
im73

Test run stop button

  1. Confirm stopping.
  2. The test is passed.
im74

Test is passed

  1. Look at the test agent computer. The folder C:Users is opened and the โ€œ1โ€ folder is present there.
im75

Test agent

1.12 Alternative way to assign test cases to test methods.

Exporting test cases from UI Test Methods.

Look for details: http://msdn.microsoft.com/en-us/library/dd465191.aspx

Instead of creating test case and then associating it with test assembly, you can create a test case from the test assembly.

  1. Open the Visual Studio command prompt: Start -> Microsoft Visual Studio 2010 -> Microsoft Visual Studio Tools -> Visual Studio Command Prompt 2010.
im76

MS VS command prompt

  1. Change directory to the folder that contains the assembly:

cd  C:Usersadministratordocumentsvisual studio 2010ProjectsTestProject1TestProject1binDebug

im77

Chage folder

  1. Import all the test method to test cases:

tcm tescase /collection:http://tfs:8080/tfs/DefaultCollection /teamproject:TestProject /import /storage:TestProject1.dll

im78

Imported tests

  1. Open the Team Explorer -> Test Project -> Work Items -> Team Queries -> My Test Cases. Double Click My Test Cases.
  2. There are test cases, which names correspond to the test methods names.
im79

My test cases

The system we deployed and configured, allows to create and execute Unit Tests and Coded UI tests on the remote virtual machines. But still you need to start virtual machines and tests manually. To configure automated test run, we must install and configure SCVMM Server. Read about it in our next article!

Learn more software QA tips from the artcile describing software impact testing.

Have a question?

Ask our expert!

Tell us about
your project

...And our team will:

  • Process your request within 1-2 business days.
  • Get back to you with an offer based on your project's scope and requirements.
  • Set a call to discuss your future project in detail and finalize the offer.
  • Sign a contract with you to start working on your project.

Do not have any specific task for us in mind but our skills seem interesting? Get a quick Apriorit intro to better understand our team capabilities.