In my previous post I talked about Silverlight 2 unit testing, and I decided the next natural step was to figure out how to integrate Silverlight testing in a continuous integration process. Personally I’m a big fan of this development practice, and have practiced it with great success on several projects. One of the challenges of integrating Silverlight tests into a CI process is the fact that the Silverlight test framework runs inside the browser and you cannot write Silverlight tests using your favorite test framework like NUnit or MSTEST. The power of CI is to build your software as new code is added to the source control, run automated tests, package the software and deploy it to a testing environment. If the tests fail you stop the deployment and alert the development team.

There are several tools you can use to set up a CI server. Personally I use CruiseControl.NET, but CI is more about a practice than a specific tool or technology. CruiceControl.NET supports several testing frameworks. For this example I’m going to use NUnit. When including Silverlight tests into your build process you need a way to fail the build if one of the Silverlight tests fails. To do that I’m going to launch the Silverlight tests from NUnit, and use WatiN to have NUnit “look inside” the browser to see if the Silverlight tests succeeded or failed.

WatiN (Web Application Testing in .NET) is a framework inspired by WatiR (… in Ruby), and is basically a way of automating Internet Explorer or Firefox. WatiN gives you a simple API to control IE at run time. You can point it at an URL, navigate the HTML DOM, enter text into text boxes and click buttons. As you automate real usage of your web application you can do assertions to verify that your web application behaves like you expect. I’ve previously used WatiR for doing "smoke tests" of an application, but this is my first experience with WatiN and I’m pretty happy with the framework so far. To give you a quick feel for the framework and what you can do I’ve included a really simple test for searching for my name at Live.com.

As you can see the WatiN API is straight forward, you can grab hold of form elements and interact with them. Since the Silverlight unit tests are executed inside the browser we can use the same technique to navigate to the web page containing the Silverlight tests and inspect the browser to determine of the test was a success or not.

In this example test we’re not checking each individual Silverlight test; we’re simply failing the NUnit test if any of the Silverlight tests fails. This is good enough in most cases, but it would be nice to get some additional information about what went wrong when executing the tests from NUnit (or inside a build server). The Silverlight testing frameworks currently doesn’t provide any hooks to extract this information, but using WatiN we can reach into the HTML doom and pick out the pieces of information we’re interested in.

When running the tests from the NUnit test runner we can now look in the Console.Error section to view additional information about failing Silverlight tests.

Now that I have Silverlight tests failing NUnit it’s straight forward to include the NUnit tests in my CruiseControl.NET project configuration. When a Silverlight test fails, the entire build fails and the developers will get notified. If the Silverlight app is part of a larger web application you can use WatiN to do functional testing of other aspects of your web application as well, to give you another layer of testing in addition to "traditional" unit tests.

As Silverlight 2 Beta 2 finally shipped this weekend I’ve upgraded my YouCard demo to Beta 2. You can also execute the unit tests your self if you want.

Jeff Wilcox, who worked on the Silverlight testing framework, wrote a post about unit testing in Silverlight 2 Beta 2. In essence you can continue to use the testing framework that shipped with the Silverlight 2 Beta 1 controls, but you need new project templates (if you don’t want to set up testing by hand). He also mentions that there will be an update to the testing framework. Jeff also hints that there will be support for CI in Team Foundation Build and CruiseControl.NET. I did some Facebook messaging with Jeff about this, and I think his approach is going to be quite different to mine, so it will be interesting to see his take on CI.