Approval Tests: 2011

Saturday, December 31, 2011

Using Reporters in Approval Tests

Today I pushed new versions of ApprovalTests for both C# and Java to SourceForge. Updates include new capabilities around the Reporters feature set for ApprovalTests. These enhancements were driven by feature requests from Peik Aschan(@peikas). He and I pair-programmed remotely (he lives in Finland) to extend ApprovalTests per his ‘asks’. If you are using ApprovalTests and have an idea for a feature, please ping me via twitter @LlewellynFalco and I’ll pair with you to code up your idea as well.

Why Use Reporters?

I assume that you have tried out Reporters if you are reading this blog, however, just to get us started, I’ll remind you that a Reporter is called when an ApprovalTest fails. The three reasons for using different types of reporters are as below.

Visualizing Results

The output of a failed ApprovalTest can range from simple (a text file) to complex (*.png, *.html, *.mp3, etc…) If you get a *.html file, then you do not usually want to see the ‘page source’ output, rather you can usually more easily view (and approve) a HTML-rendered view. Because of this, you might want to decorate your class or method with

[UseReporter(typeof(FileLauncherReporter))]

Comparing Results

Of course, not only do ApprovalTests Reporters tell you the results, but also the results can help you determine what has changed since they last passed. While the screenshot above will help you to understand what the web page LOOKS like, it won’t help you to figure out why the test is no longer passing. To get more granular information, you may want to use a different Reporter, such as the DiffReporter. Output is shown below.

[UseReporter(typeof(DiffReporter))]

Approving the Results

The last reason to use a particular Reporter is to make the ability for you to approve the dersired output of the test easier. The DiffReporter is most often used for this, because you can just right click in your particular differencing tool (TortoiseMerge is shown above, BeyondCompare, WinMerge or KDiff can also be used) and approve the output ‘approve whole file’.

In this release of ApprovalTests, I’ve added a new reporter to increase the flexibility of approving. That reporter is called the ClipboardReporter and here is how it works. Rather than launching the output in any tool, this reporter creates the command-line output needed so that you can move the results file to the approved file quickly. It automatically adds this command to your clipboard:

[UseReporter(typeof(ClipboardReporter))]

Simply open a command prompt and past the contents of the clipboard in to the command prompt window to approve the file.

How to use Reporters

In the release, we have added the ability to decorate an assembly (C# only), class or method with multiple reporters. This is easy to do, just add the reporters of interest to the level, separated by commas. The current list of reporters is as follows:

C# Reporters	Java Reporters
BeyondCompareReporter.cs ClipboardReporter.cs DiffReporter.cs FileLauncherReporter.cs ImageReporter.cs MsTestReporter.cs MultiReporter.cs NotepadLauncher.cs NUnitReporter.cs QuietReporter.cs WinMergeReporter.cs	ClipboardReporter.java DiffReporter.java EnvironmentAwareReporter.java ExecutableQueryFailure.java FileLauncherReporter.java FirstWorkingReporterChain.java GenericDiffReporter.java ImageDiffReporter.java ImageWebReporter.java JunitReporter.java MultiReporter.java NotePadLancher.java QuietReporter.java TextWebReporter.java TortoiseDiffReporter.java WinMergeReporter.java

Here is the code example for the scenario described above, i.e. both HTML (browser) and Diff (source comparison) using Tortoise Diff.

1: using ApprovalTests.Reporters;

   2:  using NUnit.Framework;

3:

   4:  namespace ApprovalTests.Tests.Html

   5:  {

   6:      [TestFixture]

   7:      [UseReporter(typeof(DiffReporter), typeof(FileLauncherReporter))]

   8:      public class HtmlTest

   9:      {

  10:          [Test]

  11:          public static void TestHtml()

  12:          {

  13:              Approvals.ApproveHtml("<html><body><div style='font-family:Broadway;font-size:18'> Web Page from ApprovalTests</div></body></html>");

  14:          }

  15:      }

  16:  }

I have found that using the right Reporter or Reporters at the right time in the testing cycle has made me more productive. Let me know how it works for you.
Happy testing!

Thursday, July 21, 2011

Testing Rdlc Reports

[Download ApprovalTests]

Edit [4/25/2012] There is now a video on testing Rdlc Reports as well

Rdlc reports are a very common thing with many .Net projects. They are a convenient way to both show data, and produce printable pages. I have seen many of these when working with legacy code, and wanted to share how to test them.

Approval Tests offer a wide array of convenience methods to test rdlc reports. I’m going to build from the most common, to the most robust.

Scenario 1: Rdlc report, backed by a single dataset, all in the same project.

This is probably 95% of existing rdlc reports. In this example I have a project with rdlc report printing out the best insults from the insult database we use to teaching kids T-SQL at www.teachingkidsprogramming.org

It is all in a single project, the rdlc file is an embedded resource and then uses a Dataset called InsultsDataTable.

The great thing about testing rdlc reports is with approval tests is it’s still the single line you would expect....

Approvals.ApproveReport( reportName, reportData);

The report name is easy, it’s just a string for the location of the embedded rdlc file. The reportData might be a bit harder to recreate. Of course you can always dummy up the InsultsDataTable object by hand, if you are particular about the text of any given field, but I find often I am not, so we added a connivence extension method to DataTable to automatically add a dummy row to it. If you want multiple rows, just state how many. For example: .AddTestDataRows(42)

Here’s the Code

Approvals.ApproveReport("ReportingDemo.InsultsReport.rdlc",
new InsultsDataTable().AddTestDataRows());

when you do this, it will create render the rdlc report to a tiff file (I usually use the DiffReporter for viewing) and show you the output, simply approve it (I usually copy the move command from the console and run it in a command window).

Scenario 2: Being explicit and the datasource name

There was a lot of assumed knowledge in the previous step, let’s start with the name of the datasource. What if you want to state the name directly? Well, it’s basically the same code, except that you will now add the name

Approvals.ApproveReport( reportName, reportDatasourceName reportData);

Of course, who can remember those names ( I always try to fill in “Model” for mine to have uniformity, but then again we are talk about legacy code which is rarely uniform). What I usually do is pass in a blank string “”, and let the error message guide me.

Approvals.ApproveReport("ReportingDemo.InsultsReport.rdlc", "",

new InsultsDataTable().AddTestDataRows());

This will produce the following exception:

System.Exception : The Datasource Name '' is not a legal match for ReportingDemo.InsultsReport.rdlc, Legal Matches are: [Model]

See, this report datasource should have been “Model”. This is especially useful when there are multiple datasource (below).

Approvals.ApproveReport("ReportingDemo.InsultsReport.rdlc", "Model",

new InsultsDataTable().AddTestDataRows());

Scenario 3: Rdlc is in a separate assembly from the datasource

So far we have been assuming that the datasource is in the same assembly as the embedded rdlc file. This is not always the case. If they are in different assemblies, we need to specify which assembly to look in. There are a couple of ways to do get the assembly, but I prefer using a class I know to be in the same assembly.

typeof(KnowClassFromAssembly).Assembly

again, here's the method signature:

Approvals.ApproveReport( reportName, rdlcAssembly,

reportDatasourceName reportData);

Scenario 4: Multiple Data Sources
The last case is when there are multiple datasource, in this case you need to make tuples for each pairing.

Approvals.ApproveReport( reportName, rdlcAssembly, reportPairing[]);

Here's the code:

Approvals.ApproveReport("ReportingDemo.InsultsReport.rdlc",
typeof (KnowClassFromAssembly).Assembly,
Tuple.Create("Model", new InsultsDataTable().AddTestDataRows()),
Tuple.Create("Address", CompanyInformation.Instance));

So now you have everything you need to easy get those rdlc reports under tests.

Have fun testing!
Llewellyn Falco

Note: there is a slight variation between the Page size of a PDF and a multipage Tiff.
If your report is very tight to the page, the page rendering might be different.

Sunday, July 3, 2011

Testing Asp.Net pages

Update - Arlo Belshee pointed out this applies better to "older" style asp programming. If your pages don't resemble this pattern, please check out his response

[Video at bottom of page]
Here is the strategy I’ve been using for unit testing Asp.Net pages. Aspx pages were almost intentionally made to be hard to test, so the result is as elegant as is possible for the given situation.

Note: If you want to test something that isn’t the “end result” of a web page, simply pull that logic into a separate class\method\dll and test it normally. This is for testing the final output of the page.

The Bottom Line: The following 1 line will test the AspxClass for a given scenario.

Approvals.ApproveAspPage(new AspxClass().TestMethod)

Architecture:

The Normal architecture when testing web pages is as follows

The strategy I've developed is very simular to this, but allows for better "test functionality" by adding a test method IN the aspx page, and then allowing a secret handshake, or backdoor to the aspx page to run that scenario.

Most Unit Tests for an asp.net page will take this form
1) Create an aspx page
2) Do some scenario
3) Verify the output of the page

When you do this in your testing, those 3 steps are going to be split out across your code into 3 separate places:
1) The PageLoad event of your aspx page will handle the creation of the page and the redirect to the scenario of your choice (the TestMethod).
2) There will be a TestMethod which does the scenario part IN your aspx code behind page.
3) The Unit Test will Callout to the aspx page and verify the html output from the page. This actually starts the whole thing.

Simple Example:

Let’s say I wanted to test how an aspx page renders for a given company invoice:

Step 1) Write the Unit Test verification IN the Unit Test Class

Approvals.ApproveAspPage(new CompanyInvoiceView().TestSimpleInvoice);

it’s important to note that I’m not calling the method “TestSimpleInvoice” this is just a Delegate to that method. This is so I can “reflect” the http call needed to run the test.

Step 2) Write the Unit Test Setup IN the aspx page

public void TestSimpleInvoice()
{
invoice = new Invoice{Name = “Test Company”}
invoice.AddLineItem(“candy bar”, count = 2, cost = 0.50);
invoice.AddLineItem(“soda”, count = 1, cost = 1.50);
invoice.TaxRate = 0.10;
}

Step 3) Adding the “Test Diverter” IN the aspx page

Next, we need to intercept the call to go to the Test Method. This means adding the following code at the beginning of the Page_Load event

protected void Page_Load(object sender, EventArgs e)
{
if (AspTestingUtils.DivertTestCall(this))
{
return;
}

....
}

Step 4) Turn on the local test server and then run the tests.

Here’s a video of the whole thing in action...

Friday, April 22, 2011

Illusions of Grandeur and Agreement

I just finished watching Adam Savage presenting at maker faire. It’s a pretty great talk. Kinda.

Here’s what I’m taking away:

“Set deadlines for personal projects”

But about 22 minutes in I become pretty dissatisfied, and I want to talk about why:

I am agreeing with Adam about everything.

Everything? Yep, It ALL makes sense. I know it ALL already... Now do I believe that if I was to sit down and actually do something with Adam that everything would be great? No disagreement? Nothing to learn?

Of course not. That’s a stupid idea. There would be tons to learn probably in the first 22 minutes. So why am I “sitting down with him now” and not disagreeing? I think my girlfriend, Lynn Langit who happens to be a Microsoft Evanglist, said it best today:

“Evangelist seek to make everyone in the room feel smart,

Trainers try to make everyone in the room be smarter”

This illusion of agreement is starting to bug me more and more. I don’t want to spend and hour feeling smart. I want to be smarter, but I think I’m a bit odd in this. Odd not just in the desire to learn, but in my reaction when I am, or am not, learning.

There is an interesting video from Veritasium about this.

http://www.veritasium.com/2011/03/khan-academy-and-effectiveness-of.html

Learning involves some disagreement. So next time you are watching someone talk, ask yourself “Am I agreeing with everything?” If you are there is probably a miscommunication going on.

I am constantly reminded of the training scene from ‘The Matrix’:

Morpheus: How did I beat you?

Neo: (breathing hard) You... you're too fast.

Morpheus: Do you believe that my being stronger or faster has anything to do with my muscles in this place?

Neo: (Shakes Head No)

Morpheus: Do you think that's air you're breathing now?

Neo: (Stops breathing hard)

Saturday, January 8, 2011

Lambdas v.008, usable Now! Forward compatible with Java 1.7

I released Java Lambdas v.008 which allows for forward compatibility with the new Lambda implementation coming down the pike for Java 7

The idea is this. Create methods that receive a Function[0-9] or Action[0-9], and then create that lambda with either my current lambda implementation F[0-9] or A[0-9] now, or when Java 7 finally implements lambdas in the syntax, simply convert them to that official syntax #{a->a}

Let's look at an example:
Get all the people over 35 years old
To do this, we want to use the method

Query.where(List, Function1<T, Boolean>)

S1 implements Function1, so we can write the following in Java 6

List<Person> people =
Query.where(getStudents(), new S1<Person>(a){{ret(a.getAge() > 35);}});

However, once Java 7 releases the official Lambdas, we can convert this to the much simpler call

List<Person> people =

Query.where(getStudents(),#{a -> a.getAge() > 35});

Because Function1 is a SAM interface (single abstract method)

Why Use Lambdas?

Hopefully this will make it easier to start using lambdas in your production code. Lambdas(Closures) are all about using the open/closed principal . They are a bit tricky to use at 1st. But soon they will become essential to you. I think of them as the next step in subroutines.

In simpler languages, you could create a subroutine, but not pass variables in. If you wanted to have a function

People.whereMimimumAgeIs21()

you could, but you could not have

People.where(int MimimumAge )

and call

People.where(21)

Parameters allowed you to reduce duplication by saying "this variable can be different for each call".

Lambdas do the same thing, but with duplicate code. Now you can say

People.where(Function1 whereClause )

and call

People.where(#{p -> p.age > 21})

Note: For the record, I dislike the currently proposed (1/8/2011) Java 1.7 lambda syntax, and vastly prefer the C# syntax

People.where(p => p.age > 21)

Links:

Download the Lambdas jar

Using Lambdas & Query

Introducing Java Lambdas

Approval Tests