How can I test the weather? We have a project where we need to know the weather report. This is possible thru a web API that will give us the weather report. However, we have no control over either the API or the weather. So how do we test this? The traditional approach is to encapsulate all calls to the weather API and then mock it out. The unfortunate part is this doesn't actually test the weather at all. It 'only' allows us to test the entire rest of the app in hopes that the encapsulated piece will work. Let's take a look at a new way to actually test the Weather API using approvals.
Feedback, Regression & Granularity :Unit Tests bring us a variety of things that are important in writing software: Feedback as we are writing the code, to know that we are writing what we wanted; Regression Testing so that once it works, it doesn't get bugs; and Granularity of Feedback, so when something goes wrong, we know exactly where and why it did.
A new approach. Let's look at a new approach that can get us all of these things, quickly and easily.
Feedback: First, we are going to need an encapsulated piece that loads the weather.
This is going to be a small class with two functions.
When we run this it will fail as all approvals do the first time and we will see the results of using this API.
So now we have the feedback that our API is working.
Regression: Of course now that it's working we need it to work from this point on. Since we have no control over the weather, when we approve this test the only thing that will be approved will be the query, not the result. From now on as long as the current query matches the old query this test will pass. This test will pass without actually running the query against the web service.
This is important so we'll say it again, The test passes if the query hasn't changed.
This means the approval artifact would look like this :
GetCityWeather?city=San%20Diego
Granularity When Things Change: Of course all things change. Let's say in the future someone changes this call so that it returns the temperature in Celsius. Lets take a look at what would happen when you run that test.
This is going to be a small class with two functions.
class WeatherLoader
{
getQuery(); // returns the query that's used to get the weather
execute(query); // executes the given query to get the weather
}
Now we're going to write a test that exercises this.{
getQuery(); // returns the query that's used to get the weather
execute(query); // executes the given query to get the weather
}
Approvals.approve(new WeatherLoader("San Diego"));
When we run this it will fail as all approvals do the first time and we will see the results of using this API.
Query:
GetCityWeather?city=San%20Diego
Result:
<WeatherReturn>
GetCityWeather?city=San%20Diego
Result:
<WeatherReturn>
<Success>true</Success>
<ResponseText>City Found</ResponseText>
<State>CA</State>
<City>San Diego</City>
<Temperature>62</Temperature>
<Unit>Fahrenheit</Unit>
</WeatherReturn><ResponseText>City Found</ResponseText>
<State>CA</State>
<City>San Diego</City>
<Temperature>62</Temperature>
<Unit>Fahrenheit</Unit>
So now we have the feedback that our API is working.
Regression: Of course now that it's working we need it to work from this point on. Since we have no control over the weather, when we approve this test the only thing that will be approved will be the query, not the result. From now on as long as the current query matches the old query this test will pass. This test will pass without actually running the query against the web service.
This is important so we'll say it again, The test passes if the query hasn't changed.
This means the approval artifact would look like this :
GetCityWeather?city=San%20Diego
Granularity When Things Change: Of course all things change. Let's say in the future someone changes this call so that it returns the temperature in Celsius. Lets take a look at what would happen when you run that test.
| Received | Approved |
| GetCityWeather?city=San%20Diego&unit=Celsius Result: <WeatherReturn> <Success>true</Success> <ResponseText>City Found</ResponseText> <State>CA</State> <City>San Diego</City> <Temperature>29.4</Temperature> <Unit>Celsius</Unit> </WeatherReturn> | GetCityWeather?city=San%20Diego Result: <WeatherReturn> <Success>true</Success> <ResponseText>City Found</ResponseText> <State>CA</State> <City>San Diego</City> <Temperature>85</Temperature> <Unit>Fahrenheit</Unit> </WeatherReturn> |
Both queries are re-run: Notice that the temperature of the approved file is 85 not the 62 when it was originally approved. This is because the query is the only thing that is persisted. The result of the query is re-executed at the time of failure giving us the current temperature in the approved section.
Then notice that the query is re-executed for the received side. Through the beauty of diff tools not only do we get the large picture of what's going on but the details and granularity of everything that has now changed.
Testing the Indeterminate : This is a one line test that runs against an external Indeterminate system. It required zero setup and we have zero control over it, yet, in the end we have tests that give us Granularity, Regression and Feedback.
Other possible cases: Databases are another place where being deterministic can be a problem. Of course you can do a large amount of setup to force them to be deterministic. And, of course, this carries a high price in both writing the test and maintaining it. Let us suggest that you just accept that things aren't deterministic and use the above approach.
As a final thought. Agile was created by accepting that requirements change. This approach was created by accepting that not everything we test is deterministic.
No comments:
Post a Comment