Talk at XPSD Thrusday Nov 6th

we are giving a talk on approval tests at XPSD Thursday Nov 6th at 6pm, here's the teaser...

Approval Based Tests

Write your code, run it, see that it works. This is a familiar process to all programmers. The addition of 1 line of code can now make that a repeatable automated regression test!

Well tested code has been shown, over and over, to improve the quality and maintainability of software. Yet most projects are not well tested. Why is this? Too often programmers feel that testing imposes too high of an overhead on writing code.

In this talk, our goal is to make it easier and faster for you to write unit tests.

We will show how using approvals, in addition to asserts, will great increase not only the ease and speed of writing a test, but the completeness and maintainability of those tests; bridging the gap between people who don't write tests, and test driven developers.

"I feel like I have to write my code twice…" Right now, unit tests impose a difficult step upfront of knowing exactly what you want. Asserting this outcome then becomes cumbersome the more robust & detailed it is. After that, maintaining the test becomes equally cumbersome. Because of this, tests either do not get written, or get discarded, leaving untested code which is fragile, error prone and leads to the nightmare of maintenance so many experience.

After this talk, you should find yourself spending less time writing tests, yet having better tested code, a proven way to increase successfulness of your software.

"After I heard Dan & Llewellyn talk about approval tests, I tried it out… in the first 4 minutes I was testing my code" - C. Monkey

"I find myself writing more code just so I can write more approval tests!" - Script Kiddie

"All my life I'd sought approval from other developers, now I can approve on my own" – Noself E. Steam

"With all the time I saved, I actually learned to talk to women" - Mystery

Approval Artifact Files

One key element of approvals is the approved artifacts that are created, and I wanted to take some time to go into these. First off, it is worth mentioning that for xUnit type tests, what is created is a file of the name

MyClass.myMethod.approved.extention
... and ...
MyClass.myMethod.received.extention

This file resides in the same directory as the source file for MyClass.

Many times this is simply a text file. The output being approved being taken from the toString() method, or a similar printer. Sometimes it is an html file, which is really just a txt file, sometimes it is a Pdf file or Png image file.

Q: Are these files kept in source control?

Yes, the approved files reside in your source control. This has 2 big benefits

1) The test are repeatable on different systems.

2) You can flip back through easy to view output, to pinpoint when and where a bug got introduced.

Q: What configuration is needed?

None.

Q: How can I write to a custom file format?

Simple, all Approvals.approve(object) calls simply construct an approver, and a reporter. The approver is normally a FileApprover, which takes a FileApprovalWriter and an ApprovalNamer.

To do a custom file output simply make your own ApprovalWriter and your own convenience approve() function

Downloads on source forge!

This weekend we got the Java and C# versions packaged up on source forge. 

Download them, try them out. Tell us what you think.

https://sourceforge.net/projects/approvaltests/

I know exactly what I want!

Well...sometimes

When creating software, customers don't usually know exactly what they want at the beginning. Agile has checkpoints that allow for quick feedback; while the product is under development, customers can see features as soon as they are completed, giving them an opportunity to accept it or to suggest changes.

Approvals give developers the same opportunity. As a developer, you may not know exactly what you want. The more insight you can get from a result, the more you understand where you want the design to go. That tends to lead us to a more natural flow of coding. You start off with a design in your head, code up an example usage, and implement it. When the result is what you expect, you approve it. With your approved result, you also have a regression test that ensures your expectations don't get violated.

Here's an example test from a bowling game without approvals:

public void TestFourThrows()
{
game.Add(5);
game.Add(4);
game.Add(7);
game.Add(2);
Assert.AreEqual(18, game.Score);
Assert.AreEqual(9, game.ScoreForFrame(1));
Assert.AreEqual(18, game.ScoreForFrame(2));
Assert.AreEqual(3, game.CurrentFrame);
}

This is nice; we can see some of the usages for the game object. Now lets take a look at the approval version:

public void TestFourThrows()
{
game.Add(5);
game.Add(4);
game.Add(7);
game.Add(2);
Approvals.Approve(game);
}

The approval line doesn’t show us any usages of game like the first example. Glancing at the code we don’t even know what we expect either. When we look at the content of the approval we see:

Frame 1: 9
Frame 2: 18
Current Frame: 3
Total Score: 18

or

|  1  |  2  |  3* | Total |
|  9  |  18 |     |   18  |

This output is much more expressive and gives us an understanding of the state of the game object. It allows us to change our internal design without having to change our test code. As we move forward we may find the need to add more information to the output. We make the change, fail the approval, then look it over; if we like the new result we approve it.

Those of us who push a design by writing tests first may have issues with this approach. Instead of creating assertions against primitive types for our expectation just trust the one in your head. While we code our expectations might change. Again, Approvals make it easier for us to change our code without changing the test code. As a discipline we still need to work in small units and design as we go. Our code can also skip those rare artifacts we sometimes need to create for testing. Because we've spent the time up front working out a visual for our code we can easily verify it.

When it comes to pairing, this allows both developers to come up with a common representation of the result they expect. When something fails, instead of double checking if our assertions were correct, we can look at the approval and discuss the expectation as a whole. We spend more time talking about the design instead of how to test it.

There are many ways to test code and they provide different solutions. Some methods push a design, some confirm them, some are used for regression testing. When you write tests, you need to decide if describing the interface is more important or the outcome. Approvals is another tool to help you write better code and get you closer to exactly what you want.

Approve is the new Assert

assertEquals(5, person.age());
assertTrue(person.isFemale());
assertEquals("jane", person.getName());

Asserts are a great way of checking primitives. However, they get cumbersome when you want to assert an object. Even the above example really is asking for

assertPerson(person, "jane doe, female age 5");

Of course this quickly becomes a nightmare of trying to describe the person object. This is where approvals come in.

Approvals.approve(person.toString());

Now you can get a test with nicely formatted output

Name : jane doe
Age : 5
Sex : Female

The more complex the object, the greater the task of writing & maintaining the asserts becomes. While approvals always stay the same one line: 

Approvals.approve(yourObject);

So whether it’s a single object, a tree of objects, a Gui Component, an Html page, a pdf document, an email, etc… if you can output it to a file, you can approve it. Then you can always view that output in whatever manner you wish. So there’s no need to find a fancy way to describe a pdf: just view it in acrobat, and if it’s good approve it.

Approval Tests (a pictures worth a 1000 tests) [repost]

Scenario 1 [Testing a User Interface]

Core Concept :

Component gui = createGui();
Approvals.approve(gui);

This creates a snap shot of the gui: gui.received.png

This passes if gui.received.png == gui.approved.png

If you like the result, simply rename gui.received.png -> gui.approved.png and the test will pass.

Could you explain that in a diagram?

yes.

Why is this awesome?

“a picture is worth a 1000 tests”. Ever find that your unit tests aren’t giving you enough security? Have you started making lots & lots of asserts? Ever find your tests test too many things and the maintenance required to change anything is actually reducing your agility?

Approval Testing gives you 100% lock down on your gui, yet changes simply require a click of a button.

Where are the approved images stored?

Approved images are stored in your testing folder and in your source control. This makes the test repeatable on different systems & you can actually browse a visual of how your gui’s have changed over time!

How does it work for a web app?

String html = createWebPage();

Approvals.approve(html);

Instead of an image, now a text file (page.received.html) is created.

Scenario 2 [Behavior Driven Development]

Core Concept :

Object myObject = createObject();

/*temp*/ Approvals.approve(myObject.toString());
doSomething1();
/*temp*/ Approvals.approve(myObject.toString());
doSomething2();
/*temp*/ Approvals.approve(myObject.toString());
doSomething3();
/*temp*/ Approvals.approve(myObject.toString());
doSomething4();
/*temp*/ Approvals.approve(myObject.toString());
doSomething5();
Approvals.approve(myObject.toString());

As the code is created, the approval will move down, so That the final code is

Object myObject = createObject();
doSomething1();
doSomething2();
doSomething3();
doSomething4();
doSomething5();
Approvals.approve(myObject.toString());

Why is this awesome?

Many times a unit test is scattered with spot testing to assure things are done correctly. This creates a barrier to readability, and can lead to unneeded code.

Approval tests reduce the need to spot check, making the test cleaner and simpler.

What if I don’t want to change my toString?

No problem, create a toOtherString(), or a ObjectWritter, or Inspect the object. Sometimes we even create a ObjectVisualizer when an image is a better way to show the state.

What if there’s a bug in my code?

The first thing to do is to try to walk through your code and get an idea of the state of the object, so you can see where it went wrong... if only you had a nice way of visualizing the state of your object.... oh wait! that’s the first thing you created! I guess it will be pretty easy to debug.

Scenario 3 [The lock down]

Core Concept :

Say I have a web page that displays a user profile. I want to refactor it, but I have no tests. However, I do have 300 users in my current database.

String allPages = getAllUserProfilePages();
Approvals.approve(allPages);

Why is this awesome?

2 lines of code to completely lock down a legacy process. That is the definition of awesome!

Won’t this test be kinda slow?

Yes. But after you are done safely refactoring, you can/should remove the tests.

Is this test data independent? Will it be repeatable tomorrow?

No. But after you are done safely refactoring, you can/should remove the tests.

Should I keep this test after the refactoring?

No!

Approve(this.blog)

We wanted a centralized place to put all of our thoughts on how to use approval tests.
Here it is.