Friday, October 1, 2010

How to: Run Coded UI Tests from a Command Line

This blog post is a third in a series on Coded UI tests and have assumed you’ve looked at the previous two posts:

Part 1: Create a Simple Visual Studio 2010 Coded UI Test

Part 2: Coded UI Tests: Test Lists, Test Categories, Oh my!

Now that we have a test suite and we have organized our tests, how can we run them from a command line? Once we have tests running from a command line, we can then wire it up to a continuous integration (CI) server like Cruise Control.Net (which will be a later post) and have them to run automatically.

Running a Coded UI Tests from Command Line Based on a Test List

1. Open up with Visual Studio 2010 Command Prompt Window (Start->Programs->Microsoft Visual Studio 2010->Visual Studio Tools->Visual Studio Command Prompt (2010))

2. Point to the directory of your Solution file. In our case it will be C:\Projects\GUIPOC



3. Enter the following command prompt to run all tests
Mstest /testmetadata:CalcCodedUITest.vsmdi /testlist:LoadCalcTestList

4. The test list will load and execute like the following:




Other popular MSTest commands to run tests:

These examples are based that steps 1 and 2 were followed above.

Run Test by Test Name
  • Mstest /testcontainer:CalcCodedUITest\bin\debug\CalcCodedUITest.dll /test:LoadCalculator /unique
Run Test by Category
  • Mstest /testcontainer:CalcCodedUITest\bin\debug\CalcCodedUITest.dll /category:CalcCategory
Run Test by Category and Publish Results to a Declare File
  • Mstest /testcontainer:CalcCodedUITest\bin\debug\CalcCodedUITest.dll /category:CalcCategory /resultsfile:”C:\LogFile.trx”
Tips and Tricks
  • The “.vsmdi” file stores the links to other tests. This is associated with the solution so you have the ability to store tests from multiple projects in the same solution under one test list.
  • If you declare multiple tests to run in one command line, the order in which they run are not guaranteed. If you need to ensure the order, ensure it using the command line by having multiple command lines executing tests.
  • If a results file already exists, another test run using the same results file name will error out and complain that that file already exists.
 
Additional Resources:

1 comment:

  1. Thanks Matthew, here's a handy single line cmd file for running tests (just substitute YOUR_*):

    %comspec% /k ""C:\Program Files\Microsoft Visual Studio 10.0\VC\vcvarsall.bat"" x86 ^&^& cd C:\Users\YOUR_USER_HERE\Documents\Visual Studio 2010\Projects\YOUR_PROJECT ^&^& Mstest /testmetadata:YOUR_PROJECT.vsmdi /testlist:YOUR_TEST_LIST ^& pause

    ReplyDelete