My current project is a scrum project. We develop new features in sprints and deliver working software every 3 weeks. To keep the pace steady we have loads of tests that run every night after a build is deployed.

Until recent we used ‘dumb’ powershell scripts to run tests and use the write-error to report failure. This made us aware of errors every morning so we could fix it as soon as possible. Problem was that the first error stoped the complete script and the logging was loads of plain text. Finding the error message was hard, finding the failing test was even harder.

Last week we introduced Pester as a powershell testframework. The main feature we wanted to use was the reporting of all tests in TFS2017. This would help us pinpoint the error and give a good description of what was wrong. We ended up making the regression test itself more readable in the proces.

Get it

Pester is default part of Windows 10. That is the way we got it. A full installation and update guide is available on the github page. On the build agent we load the module from source control.

Script it

I pair-programmed the Pester scripts with a tester. We took an existing powershell script and added the Describe on the first line. After that we used the Context keyword, because we needed some setup in the database to be done. Last we replaced the write-error statements with an It block that contained a Should Be. Read the documentation for details. We converted all our scripts within a day.

Sample Pester file

Describe "Loading data" {
   Context "Cleaning tables and load file" {
      # truncate tables
      # load file
      It "City table is filled with data" {
         # get record count from table
         $result | Should BeGreaterThan 0
      }
      It "Product table is filled with data" {
         # get record count from table
         $result | Should BeGreaterThan 0
      }
   }
}

Run it

Our nightly build loads the Pester module and than starts the tests with Invoke-Pester. We supply the name of the overall testscript that specifies the parameters for every test/script. With OutputFile and OutputFormat we save the test results to an xml file.

$result = Invoke-Pester ./regression.ps1 `
   -OutputFile pester.xml `
   -OutputFormat NUnitxml `
   -Quiet `
   -PassThru
if ($result.FailedCount -gt 0) {
   Write-Error "Some tests failed"
}
Exit $result.FailedCount

TFS2017 has a publish test results task that can handle the xml file from Pester.

image from https://msdnshared.blob.core.windows.net/media/2016/03/TestsSummary.jpg

References

Get started with Pester (PowerShell unit testing framework)