I have previously blogged about the Silverlight testing framework that was originally released as part of the Silverlight 2 Beta 1 control source. Jeff Willcox, the lead developer on the framework, recently announced an updated version of the framework for the Silverlight 2 Release Candidate. The updated version contains several interesting new features, which Jeff has covered in a great thorough blog post. I’m not going to re-iterate the feature list in this post, but rather focus on the one of the new features; tagging of tests.
Types of automated testing
Automated unit testing of code has become a common development practice, and as developers we have a wide variety of frameworks to pick from to help us do testing. A common misperception is that any test written using a testing framework is a unit test. That is not necessary true as there are multiple other types of test you can write for your code. Phil Haack, one of the program managers on ASP.NET MVC, wrote about the boundaries of unit tests on his blog a few months back. I recommend this post as a good overview of some of the considerations you should make when writing unit tests.
I’m quickly going to list the types of tests I normally write when building Silverlight applications.
- Unit Test
Tests a single unit of work. In most cases a single method in a single class. Normally don’t touch the database, the file system or the network. Should execute quickly with no side effects (changed global state).
- Integration Test
Tests how multiple classes integrate with each-other, or how a class integrates with the environment. I.e. parsing of an XML file, reading/writing to a database, or accessing data on the network.
- Smoke Test
Simulates an user interacting with the application. Tests multiple classes and can cross boundaries such as network, file system or database. In many cases you write your smoke tests by automating the user interface.
Even if your tests aren’t a true unit tests according to the list above, it’s perfectly fine to use a testing framework like nUnit to write your integration- or smoke tests. There are other types of tests as well, i.e. performance tests, stress tests and acceptance tests. For these types of tests you often use different tools or extensions to your testing framework.
One of the considerations you should make when writing your tests is to keep your unit, integration and smoke tests separated. You should try to keep your unit test running as quickly as possible, something that is easier to achieve if you focus on not crossing any process boundaries. If testing becomes slow and tedious developers might start cheating and not run the test suite as they change code. One way to separate your tests is to have different projects for each type of tests. This is a good solution in many cases, but for smaller projects (like all of my Silverlight demo applications) this may lead to a cluttered project structure. In the Silverlight testing framework you can organize your tests in a different way by adding tags to test classes and test methods.
One of the new features that were introduced in the latest version of the Silverlight Testing Framework is support for tags. You can apply multiple Tag-attributes to any test class or method in your test suite. The Tag-attribute takes accepts a string, so it is up to you to decide how to organize your tests. As an example for this post I am going use the YouCard application I built for REMIX. The tags I am going to apply are based on the types of tests I listed above; Unit, Integration and Smoke. The YouCard test suite isn’t too big and only got 13 tests in it. But it got a little bit of everything, so it should be a good example.
The most obvious candidate for being tagged as a unit test is the value converters tests. A value is like a mini-adapter you can apply during data binding to convert from one value to another. It got two simple methods, Convert and ConvertBack, and no external dependencies.
The YouCard application uses the Model-View-ViewModel pattern, so the next set of tests to be tagged as a unit test is the ViewModel tests. The ViewModel class has two dependencies to a Flickr service and a Twitter service. Because these dependencies are implemented as interfaces and injected in the ViewModel constructor I can supply test mocks, making this a pure unit test and not an integration test.
Now that we’ve identified some unit tests it’s time to show some examples of integration tests. The two most obvious candidates are the classes responsible for accessing the Twitter and Flickr APIs. Both implementations are based on an interface, making it easy to mock the dependency when testing the ViewModel. However, we still want to write tests to verify that the code downloading and parsing the XML from Twitter and Flickr works as expected. This means accessing the network, and thus crossing the boundaries of what we consider a unit test.
The final type of test we want to identify and tag is a smoke test. The Silverlight Testing Framework runs inside the browser with full access to the Silverlight runtime, making it easy to write user interface tests. The YouCard test suite has one smoke test simulating a user entering Twitter usernames into the textbox and hitting the add-user button. The test waits about a second between each user interaction, and it uses real implementations of both the Twitter and Flickr interfaces. This makes the test fairly slow to execute, but still valuable as it gives us a verification that the application works as a whole when all the pieces are coupled together.
Executing tests based on tags
Now that the tests are categorized using tags, how do we execute specific tests based on the tags? In the Silverlight Testing Framework you create the test page in the Application Startup event in the application class. In CreateTestPage method accepts a UnitTestSettings object you can use to configure the test framework. One of the properties on the settings object is a TagExpression property. The name of the property hints that we can supply more than the name of a simple tag. Jeff hasn’t documented this feature yet, so I don’t know the limits of the expressions you can create. You can do things like “!Unit” to execute all tests except the once tagged as “Unit”. I don’t know if you can use more complex expressions and combinations of tags. But in most cases supplying a single tag should be enough.
Since configuration of the test framework is done through code it’s easy to make the tag expression dynamic. The example code looks for an initialization parameter called tag, and if found its value is applied as the tag expression. Initialization parameters are passed in when the Silverlight object is created in on the page. If you are using HTML to create the object you add the initialization parameters as a param-element to the object-tag. If you use the ASP.NET Silverlight server control you set the InitParameters property. In this example I have a Silverlight and some ASP.NET code to check the query string. If the query string contains a tag-element the value is passed to the Silverlight control as an initialization parameter. Using this technique you can simply change the URL to execute your test suite with different tag expressions applied.
To execute the code simply navigate to the page hosting the Silverlight tests. By default it will execute all tests, but if you add a tag-parameter in the query string you can control which tests to execute.
The updated version of the Silverlight Testing Frameworks got lots of new useful features. Support for tagging is one of them. Hopefully this post gives you an introduction to how you can use tags to organize your tests. The deployed version of the DiveLog and YouCard application use this technique, and if you want to see how the test framework executes tests just click one of the links below.