XP2010 and BlackBoxRecorder

This week I have been attending the 11th international conference on Agile Software Development, XP2010, in Trondheim. It has been an amazing week, filled with learning, excellent food and awesome social events, making new friends and catching up with old. I gave two lightning talks during the conference, one on UI prototyping in agile projects (the same I gave at Smidig 2009), and one on Generating Characterization Tests for Legacy Code. The talk on characterization tests was based on some work I did back in October – December 2009. I was involved in a partial rewrite of some pretty bad legacy code. One of the techniques we discovered was that we could use the slow running integration tests to code generate fast and more targeted characterization tests for the code we were rewriting.


For the last three-four months I’ve been working on an open source testing tool that streamlines the technique of generating characterization tests for your legacy code. The project is called BlackBoxRecorder and is available on Github.


BlackBoxRecorder is a tool that uses AOP-techniques to automatically record and generate characterization tests for legacy .NET code (code without tests). You apply a [Recording]-attribute on the method you want to generate tests for, and a [Dependency]-attribute on any class accessing external resources such as the file system, web services or databases.

Each method invocation of a method marked with the recording-attribute will be stored in an XML file. The XML file contains information about the method, copies of input parameters, output parameters and return values. If the method being recorded invokes any methods on a class marked with the Dependency-attribute, the return values of the dependency will also be recorded in the same XML file. This enables automatic mocking of external dependencies when playing back the characterization test.

For each method recording BlackBoxRecorder will generate a unit test that will play back the recorded XML file against the method. The return values from the method will be compared against the recorded return values using reflection. If any change is detected the test will fail. The test will serve as a change detector (which in essence is what characterization test is), notifying you of any changed behavior in the method being tested. If the change was intentional you can configure the generated test to ignore that property on next run.

The generated tests supports automatic stubbing/mocking of the types marked with the Dependency-attribute. When the test is played back, the dependency will load return values from the XML recording and return these instead of actually calling the external resource. You do not have to change your code for this to work and even static methods are supported.

I got some really good feedback on the session, and it was pretty cool that Michael Feathers attended the session. I referenced his book and quoted him a few times in the presentation, so I was really glad he enjoyed the talk.


I also got questions from several people who where interested in what it would take to port BlackBoxRecorder to Java. So on Thursday evening I organized an open space session where we walked through the code and discussed how to implement a Java version. We continued the talk over lunch Friday, and soon enough Ole Christian Rynning, Johannes Brodwall, Geir Amdal, Marius Kotsbak and myself where hacking away on a spike implementing BalackBox for Java.

Hacking away on a Java implementation of BlackBoxRecorder

We called the implementation JackBox, and the code is up on Github if you want to jump in and contribute.



I will definitely blog more about BlackBoxRecorder as soon as I’m done with my NDC2010 presentations. Thanks to organizers, speakers and attendees for making XP2010 a memorable experience!

Jonas Follesø

Programmer, Triathlete, Traveler, Under Water Enthusiast and GoPro Videographer from Trondheim, Norway. Works as Chief Scientist and Manager at Bekk Consulting AS.

Trondheim, Norway http://jonas.follesoe.no