Wednesday, September 19, 2012

What’s New in Java ApprovalTests V 0.12

In this release of ApprovalTests (available here ), we’ve only added one thing, but it’s a big thing. Specifically, it’s a big data thing.
My thanks go out to John Heintz for helping me to create this. 
Here's a video of the whole thing:

Unit Testing HadoopMapReduce

Before we go into the details of everything involved in HadoopMapReduce, let’s go through an overview of how we’ve been testing our MapReduce jobs. In this example, we are doing this common demonstration of a word count MapReduce job. The idea is that sentences will come in, they will be split into words, count them up, and summarize in our reduce job.

Visualizing the System

Unit Testing provides many different advantages to a programmer. And all of them have different importance to different people, but one of these advantages is specification. In particular, with MapReduce, being able to understand the flow and transformation of the data can be more enlightening than the other aspects of Unit Testing.
One of the things that we strive to do with ApprovalTests, is create output that is meaningful and insightful. To demonstrate this, I’m going to start output of my word count unit test.
  [cat cat dog]
-> maps via WordCountMapper to ->
(cat, 1)
(cat, 1)
(dog, 1)

-> reduces via WordCountReducer to ->
(cat, 2)
(dog, 1)
As you can see, this output is a nice way of visually understanding what comes into the job, how the WordCountMapper transforms it, and what the final result is. Let’s take a look at the code that produced this result, and then we’ll break it down into individual segments.
Here is the entire line to create that output:
 HadoopApprovals.verifyMapReduce(new WordCountMapper(), 
                                 new WordCountReducer(), 0, "cat cat dog");
Easy right? Let’s look into the parts.

MR Unit

HadoopApprovals sits on top of MR Unit . You will need to grab it and include the jars. The javaDocs for HadoopApprovals mention all of the jars needed to run it. They are:
  • hadoop-client-1.0.3.jar
  • hadoop-core-1.0.3.jar
  • hadoop-test-1.0.3.jar
  • log4j-1.2.15.jar
  • mockito-all-1.8.5.jar
  • mrunit-0.9.0-incubating-hadoop1.jar


HadoopApprovals has three main functions: the ability to test the mapper, the reducer, and the map reduce combination. To make this easier, we are going to use extra information with the generics called SmartMappers and SmartReducers. We will talk about those in the next section.

Testing a Mapper

Here’s the method to test the mapper:

verifyMapping(SmartMapper mapper, Object key, Object input)

To test a mapper, you need the mapper you’re going to test and the keyValue pair going in. Many times, mappers do not actually use the key that comes in, but regardless, you will need one anyway. Do not make it null.

Testing a Reducer

Here’s the method to test the reducer:

verifyReducer(SmartReducer reducer, Object key, Object... values)

When testing a reducer, you will need to pass in the key plus a set of values for the key. You will notice you can pass in regular strings and regular integers instead of Text and LongWritable. Again, this is because of the SmartMappers we will talk about in the next section.

Testing a full MapReduce job

Here’s the method to test the MapReduce:

verifyMapReduce(SmartMapper mapper, 
SmartReducer reducer, Object key, Object input)

As you might expect, this is almost identical to testing a mapper. Except with the extra addition of a reducer.


I wrote a blog about getting generic type information at runtime from Java. You can read it here:

The long a short of it is, if you add an extra interface to your mapping and reducing jobs, you can avoid a lot of boiler-plate code needed to state what the runtime types of the input and output are. You can do this by simply having your MapReduce jobs extend SmartMapper instead of Mapper.

“What? I don’t want to change my stuff!”

Fair point.  If you want to test against an existing mapper or reducer, and you don’t want to change it to use the SmartMapper extensions, you can always wrap it using the MapperWrapper (bonus:  fun to say out loud) or ReducerWrapper. Here’s a sample of how to do that:

SmartMapper mapper = new MapperWrapper(new WordCountMap(),

Once you do this, you will no longer need to wrap all your inputs in the writable classes, HadoopApprovals will wrap primitives into the appropriate context for you.

Thursday, August 9, 2012

What’s new in ApprovalTests.Net v.20?

ASP routing support

As community contributor, Krzysztof Ko┼║mic  said in his NDC talk
Using ApprovalTests makes what was implicit,
explicit so that the item can now be tested
Inspired by his talk, Jim Counts and I created a new method in AspApprovals to make the formerly implicit ASP routing explicit and now easily able to be tested.  Here’s an example
var urls = new[] {"/Home/Index/Hello", "/"};
AspApprovals.VerifyRouting(MvcApplication.RegisterRoutes, urls);

In the sample above, the delegate (MvcApplication.RegisterRoutes) will be invokes against a mock route collection, and then fed the URLs provided.  A sample output would look like this:

/Home/Index/Hello => [[controller, Home], [action, Index], [id, Hello]] 
/ => [[controller, Cool], [action, Index], [id, ]]

Event approvals

Similar to the example above, another aspect of code which is often hidden (or implicit) is which events are wired-up to your form.  This can be particularly troublesome when changing an existing, unfamiliar piece of code.  For example, it it easy to accidentally remove a button-click event.

Event approvals allows you to easily lock down existing events which are associated with an object or a form.  To do this, simply call the following:


This new method will only allow you to verify events which are directly associated to the object which is passed in.  Unfortunately, many times you don’t want to say ‘What are the events associated with this button"? rather you actually want to say ‘What are the events associated to this form, even though the button is on the form and the event is associated to the button (rather than the form)?’  This is now no problem, since we took this scenario into consideration and also created a convenience method for the latter scenario.

To test all the events on a form and its immediate children, simply call code as shown below:

WinFormsApprovals.VerifyEventsFor(new DemoForm());

WPF support for Controls

It’s always been possible to test WPF windows with a simple:


This would render the window to a .png image and verify against .approved file.

But sometimes you want to scope smaller to an individual control. Unfortunately, we had previously neglected this scenario – no longer!  You can now test an individual WPF control with the same call as shown below:

var menu = new ContextMenu();
menu.Items.Add(new MenuItem() {Header = "Add Element"});
menu.Items.Add(new MenuItem() {Header = "Delete"});
menu.Items.Add(new MenuItem() {Header = "Edit"});
menu.IsOpen = true;

This code will produce a *.png image, such as the one shown below:

Entity Framework support

People have long pondered how to test Entity Framework.  We have talked about this before, but previously testing EF with ApprovalTests had always required a large number of steps.  Now you can test EF by simply calling

using (var db = new ModelContainer())
    EntityFrameworkApprovals.Verify(db, CreateCompanyLoaderByName2(db, "Mic"));

This method will implement a ‘test-the-weather’ scenario which will verify the resulting SQL without the need to be able to connect to a database.  However, if this method fails, then it will connect to referenced database and will execute the generated query and will return extra information about WHY the test failed.  I plan to create more detailed tutorials on the inner workings of this new method soon.

Let’s give you one more look at this method signature.

public static void Verify<T>(ObjectContext db, IQueryable<T> queryable)

Email support for attachments

Previously, we added the method


Unfortunately, when the email message contained attachments, then the GUIDs that those attachments created caused issues with test output because they were not consistent over multiple test runs.  This bug has been fixed in this release.  To learn more about approving email messages, watch this video.

Testing Email with ApprovalTests

FileLauncher with DelayReporter

There has been a great deal of demand to use Cassini-dev after Jim Counts demo’d it during his ASPConf video on testing ASP.MVC views. Unfortunately, when using this method it’s a bit tricky to use a file launcher, since the web server dies BEFORE the browser can hit it.  We came up with a simple solution which adds a tiny delay into the method execution so that this is no longer an issue.


Well, that’s all for this release. As always, if you have any questions about ApprovalTests, tweet them with the hashtag #ApprovalTests, I monitor that and will answer you promptly.

Happy testing.

Sunday, July 22, 2012

Tooling stack of testing Asp.MVC Views

“Craftsmen know their tools”

At last weeks AspConfJim Counts did a session on Testing MVC Views with ApprovalTests.
While I am always happy to see more ApprovalTesting in the world, I was also impressed with the full tooling stack Jim used throughout the presentation.

Here’s the List
CassiniDev   Inline webserver
NCrunch Continuous Test Runner
GitHub for Windows Source Control
CruiseControl.Net Continuous Integration
TortoiseDiff Diff Viewer (part of TortoiseSVN)
Code Rush Productivity Enhancement for Visual Studio
NuGet .Net Package Manager

It was great to see this level of professionalism in a demo session; an area usually reserved for cowboy programming of hello world demos. Don’t get me wrong, Jim still does a simple “Hello World” type demo, but he applies all the rigor and professionalism I would want in a real project to that example.

Nicely done Jim, you’ve inspired me to raise my game, and hopefully others as well.

Saturday, July 14, 2012

What’s new in Java ApprovalTests v.011

Today I Released ApprovalTests for Java v.011. Here's what new:

Approvals.Approve()  => Approvals.Verify()

I finally got around to fixing an early mistake in my naming convention.  Approvals.Approve() is now deprecated, and Approvals.Verify() is the new way to verify your result.

Mac OsX Diff Reporter Support

To help support my friends on the Mac Side, I have added a whole bunch of reporters for them. Including:
  • DiffMergeReporter
  • KaleidoscopeDiffReporter
  • TkDiffReporter
What do you need to do to use these? Nothing, the regular DiffReporter will cascade appropriately to find the right one on you machine.

Machine Dependent Testing

If you have multiple environments, which render things differently,  you can now use the line
 NamerFactory.asMachineSpecificTest(new OsEnvironmentLabeller());

Which will add the machine OS as part of the approved & received file name. 

For Example:  ClassName.MethodName.Mac_OS_X.approved.txt

This will allow you different Golden Masters for different Environments.

Saturday, May 19, 2012

Testing Views in Asp.Mvc

For many years, the views (the html returned from an rendered MVC controller) were an extremely difficult thing to test. Like many unreachable things, they were declared “not worth it” and ignored.

Until now,
with ApprovalTests we have created a “simple” way to test the rendered output of a view.  I say “simple” because there is a fair amount of overhead for the 1st test, which means we wanted to give you some resources to get it working for yourself.

I created a short video tutorial, and Jim Counts put together a fabulous 3 part blog + Sample Code.

Now you can decide for yourself how important it is to test the views.

Video :

Blog :

Part 1 – Getting Started
Part 2 – Working with Data
Part 3 – Tackling the Build Server


Saturday, May 5, 2012

The Testing Circle

This is the pattern I find myself in day after day while practicing TDD or BDD.

Step 1?

Usually I start at the whiteboard. It is the easiest place to start as I sketch out the scenario I want to program, but not always. Sometimes a mockup file (image, html, or xml) is provided and I start at the result, sometimes I’m give a scenario already in English, and sometimes I have to fiddle with the code first to see what’s happening.

No Beginning, No Ending

The great thing about the circle is no matter where a start, the flow should always be there in the end. If it isn’t, I take this as a smell that I am neglecting something. Also, while I wrote the arrows going clockwise (the normal way I view this process) they can equally go in reverse.

Smell 1: No Result –> Whiteboard

If the result does trigger the original image on the whiteboard (usually long erased by the time I revisit the code), then this will be a maintenance problem when I try to remember why the code looks the way it does when it changes.

Smell 2: No Code –> English

If the resulting code doesn’t make the comments so easy that they should be deleted for not adding any value, then my intention (so clear when I’m writing the original code) will not be there when my fellow programmers, or myself even a few days later, what the understand the code later.

Friday, April 27, 2012

What’s new in ApprovalTests.Net v.19?

This is a relatively small release, only 2 real changes.
  1. EmailApprovals
  2. Better Rdlc Syntax

Email Verification

Almost every site sends some sort of email. These have been hard to test, usually requiring some sort of email to be actually sent. Now you can simply say:
EmailApprovals.Verify(new MailMessage())

This is particularly nice with a


As it will open in your desktop email client (like outlook, not gmail) making it very easy to see the results. If you’ve given up on desktop clients, you can grab one from
I made a video about it here:

Verifying Email

Better Rdlc Support

Previously, if you wanted to approve a Rdlc report with multiple data sources you had to use Tuples, which got a bit annoying. In this release I am taking advantage of the anonymous enumerator syntax. The result is

   1:  RdlcApprovals.VerifyReport(reportname, assembly,
   2:        new DataPairs() { { "DataSet1", model1 }, {"CompanyAddress",model2} });


   1:   RdlcApprovals.VerifyReport(reportname, assembly,
   2:        new Tuple<string,object>( "DataSet1", model1), new Tuple<string,object>("CompanyAddress", model2);

Again, there is a video of this here:

Verifying Rdlc Reports

Happy Testing!

Tuesday, April 10, 2012

What’s New in ApprovalTests .Net v.18?

  1. FrontLoadedReporter
  2. IEnviromentAwareReport Upgrades
  3. CodeCompareReporter
  4. DictionarySupport
  5. EF and Mvc3 additions
  6. Bug Fixes

FrontLoadedReporter Attribute (For Build Systems)

You can now add the following code to your assembly.
[assembly: FrontLoadedReporter(typeof(NCrunchReporter))]

This is only allowed at the assembly level, and must be a IEnvironmentAwareReporter. If the reporter is allowed in the current environment, it will circumvent all other reporters. This is very useful for Build Systems where you want to override the behavior of the reporters.

IEnvironmentAwareReporter is file specific (better composition)

The IEnvironmentAwareReporter interface has changed from

bool IsWorkingInThisEnvironment();


bool IsWorkingInThisEnvironment(string forFile);

this allows for better composition of reporters. For example I can now combine DiffReporters that only do text with DiffReporters that do images. If you have an Image it will fall thru to the ImageReporter even if the TextDiff Program is on your system.

This is especially nice for…


My new favorite text diff tool is Code Compare, mainly because it integrates directly into Visual Studio 2010. You can download it here (it’s free)

Screen shot 2012-04-10 at 1.31.56 AM

Dictionary Support

var map = new Dictionary<string, string>(){
{"Caption", "Mal"},
{"Pilot", "Wash"},
{"Companion", "Inara"}  };

This will now create the expected approval result. I don’t know how this didn’t exist from the very start. It is worth noting that keys will be presented in their natural ordering (to allow consistency.)

EF and Mvc3

I added a Mock Saver, LambdaLoader & SingleLambdaLoader to the Enitity Framework support.
I also added .Explict to the Mvc3 Toolkit. I will make videos about this approach soon.

Bug Fixes

There were 2 bugs that got fixed this round.
64-bit machines didn’t launch 32-bit Reporters.
XUnit Reporter didn’t invoke the right Equals Method.

Both are fixed now.

Wednesday, March 21, 2012

Testing Difficult Code

I recently did 3 videos in my ‘Using ApprovalTests in .Net’ series that deal with how to test difficult code. These video actually have very little to do with either .Net or ApprovalTests, but they are common issues you run into when unit testing.


Testing difficult Code comes down to 2 things:

  1. Can I call the Method?
  2. Will the Method run to completion?

And there are 2 techniques I use to deal with these, that individually are useful, but together are amazing.

1. The Peel

The hardest part about long methods is you have to run all of it to test even a small part of it. The idea behind the peel is to break the method up into easy to run pieces.  Here’s the video:

The Peel

2. The Slice (Mocks)

Even if you get into the piece you are looking to test, you might not be able to get all the way through it. That’s where mocks come into place. By slicing off the connection to the actual trouble spot, you can easily fake it, and return to easy to run code.

The Slice

3. The Peel & Slice

So now that you have these Mocks, how do you get them to be used? While there are many forms of dependency injection (inheritance, factory, DI frameworks, IoC) the easiest of all is “Parameter passing”. Of course, to do that you need a nice seam that passes in the default dependency. Sound familiar? This is why the Peel & Slice work so nicely together.

The Peel & The Slice

Friday, January 27, 2012

Creating Namers

If your current testing framework is not supported, you can create your own namer to allow you to use it.
Check out this namer for ideas how to make your own.