Saturday, December 31, 2011

Using Reporters in Approval Tests

Today I pushed new versions of ApprovalTests for both C# and Java to SourceForge.  Updates include new capabilities around the Reporters feature set for ApprovalTests. These enhancements were driven by feature requests from  Peik Aschan(@peikas).  He and I pair-programmed remotely (he lives in Finland) to extend ApprovalTests per his ‘asks’.  If you are using ApprovalTests and have an idea for a feature, please ping me via twitter @LlewellynFalco and I’ll pair with you to code up your idea as well.

Why Use Reporters?

I assume that you have tried out Reporters if you are reading this blog, however, just to get us started, I’ll remind you that a Reporter is called when an ApprovalTest fails.  The three reasons for using different types of reporters are as below.

Visualizing Results

The output of a failed  ApprovalTest can range from simple (a text file) to complex (*.png, *.html, *.mp3, etc…)  If you get a *.html file, then you do not usually want to see the ‘page source’ output, rather you can usually more easily view (and approve) a HTML-rendered view.  Because of this, you might want to decorate your class or method with
[UseReporter(typeof(FileLauncherReporter))]

image


Comparing Results

Of course, not only do ApprovalTests Reporters tell you the results, but also the results can help you determine what has changed since they last passed.  While the screenshot above will help you to understand what the web page LOOKS like, it won’t help you to figure out why the test is no longer passing.  To get more granular information, you may want to use a different Reporter, such as the DiffReporter.  Output is shown below.

[UseReporter(typeof(DiffReporter))]

image

Approving the Results

The last reason to use a particular Reporter is to make the ability for you to approve the dersired output of the test easier.  The DiffReporter is most often used for this, because you can just right click in your particular differencing tool (TortoiseMerge is shown above, BeyondCompare, WinMerge or KDiff can also be used) and approve the output ‘approve whole file’. 

In this release of ApprovalTests, I’ve added a new reporter to increase the flexibility of approving.  That reporter is called the ClipboardReporter and here is how it works.  Rather than launching the output in any tool, this reporter creates the command-line output needed so that you can move the results file to the approved file quickly.  It automatically adds this command to your clipboard:

[UseReporter(typeof(ClipboardReporter))]

image

Simply open a command prompt and past the contents of the clipboard in to the command prompt window to approve the file.


How to use Reporters


In the release, we have added the ability to decorate an assembly (C# only), class or method with multiple reporters.  This is easy to do, just add the reporters of interest to the level, separated by commas.  The current list of reporters is as follows:

C# Reporters Java Reporters
BeyondCompareReporter.cs
ClipboardReporter.cs
DiffReporter.cs
FileLauncherReporter.cs
ImageReporter.cs
MsTestReporter.cs
MultiReporter.cs
NotepadLauncher.cs
NUnitReporter.cs
QuietReporter.cs
WinMergeReporter.cs
ClipboardReporter.java
DiffReporter.java
EnvironmentAwareReporter.java
ExecutableQueryFailure.java
FileLauncherReporter.java
FirstWorkingReporterChain.java
GenericDiffReporter.java
ImageDiffReporter.java
ImageWebReporter.java
JunitReporter.java
MultiReporter.java
NotePadLancher.java
QuietReporter.java
TextWebReporter.java
TortoiseDiffReporter.java
WinMergeReporter.java


Here is the code example for the scenario described above, i.e. both HTML (browser) and Diff (source comparison) using Tortoise Diff.


1: using ApprovalTests.Reporters;
2: using NUnit.Framework;
3:  
4: namespace ApprovalTests.Tests.Html
5: {
6: [TestFixture]
7: [UseReporter(typeof(DiffReporter), typeof(FileLauncherReporter))]
8: public class HtmlTest
9: {
10: [Test]
11: public static void TestHtml()
12: {
13: Approvals.ApproveHtml("<html><body><div style='font-family:Broadway;font-size:18'> Web Page from ApprovalTests</div></body></html>");
14: }
15: }
16: }


I have found that using the right Reporter or Reporters at the right time in the testing cycle has made me more productive. Let me know how it works for you.
Happy testing!