An alternative to bug tracking tools

One of my pet peeves is working in and with bug tracking tools. I am well aware of some of the arguments for the importance of these tools and I am not trying to address those here. Instead, I'll show you an example of an alternative that I have found useful.

First, using an approach like Specification by Example can reduce the need for bug tracking tools because communication goes up and defect counts go down. But even using this technique, defects still occasionally occur. Here is an example of how to use Specification by Example not only for 'requirements', but also for defects.

In June of this year I spoke at the Prairie Developer's Conference in Regina, Saskatchewan. Some of the speakers and volunteers were involved in creating the website, services, and mobile application for that conference. Since I was doing a Specification By Example talk I decided to use the conference web services to illustrate how easy it is to create your first automated test against a web service using FitNesse. As I was working with the services I found a small defect. Instead of writing up a defect in a bug tracker with the steps to re-produce it, I wrote a test in FitNesse to confirm the defect:

This example calls a service that returns a list of sessions and does a few basic C# calculations on that list. It counts the number of sessions (allSessions.Count) in the conference and FitNesse maps that to the NumberOfSessions variable above. It then counts the number of unique abstracts (allSessions.Abstracts.Distinct.Count) and FitNesse maps that to the Number of Unique Abstracts. FitNesse then compares the numbers and displays the results. In this case, 63 does not match 62 so it displays an error with the expected and actual results as above.

Once the test confirmed the defect, I simply communicated the failing test to the developers. When the developers reviewed the test they could clearly see what the problem was. No back and forth was required to understand the issue or to confirm the steps required to reproduce it. No one had to set the status of the defect to "working", "fixed", "duplicate", "resolved", "more information required", or anything else. One of the developers fixed the issue and even added an additional service that we could call to address the root cause - Are Session Abstracts Unique? I added the new test, ran all my tests again and was pleased to see them all go green.

This process improved communication between tester and developer, ensured that the defect would always be tested and re-tested for, and kept us from spending unnecessary time in a bug tracking tool.

FitNesse and today's date with .net

I have an acceptance test that says I need to validate the age of majority in each of the different states and provinces.  Here is a simple example:

Given Mary who is born January 5, 1995 and lives in Manitoba

When she asks if she is the age of majority

Then return no

The test above is faily simple and I could write it like this in the wiki as a Column Fixture (using the fitSharp.dll to test C# code)

!|Check Age of Majority|

|Province State|Birth Date|Am I Underage?|

|MB            |5-Jan-1995|Yes           |

The problem of course is that this test will start failing on January 5, 2013 when Mary turns 18.  Also, it does not perform the boundary testing that I would like it to do in order to test someone who is 18 today vs. someone who will turn 18 tomorrow.  In order to improve this test, I investigated some other date functions in FitNesse and a plugin by James Carr that allowed you to add days to the current date.  These work ok for smaller calculations like "Given document ABC, When it is 30 days old, Then archive it".  However, this would be a little more cumbersome for birth dates when adding 18 years (esp. with leap year calculations) and the !today function in FitNesse does not work in ColumnFixture wiki tables.  So, I found a simple way to meet my requirement.

First, I wrote a class in C# that accepts two parameters to Add or Subtract Years and Days to the current date.  The class uses C#'s simple DateTime addition to add or subtract the years/days from today and returns the result.  You could easily extend this to add months or add other functionality required in your tests:

namespace FitNesseTutorial.Tests

{

      public class GetDateBasedOnToday : ColumnFixture

      {

        public int AddYears;

        public int AddDays;

        public DateTime ResultingDate()

            {

            return DateTime.Today.AddYears(AddYears).AddDays(AddDays);

            }

      }

}

Then in FitNesse at the top of my script for this story I call GetDateBasedOnToday and store the resulting values in FitNesse variables.  Finally,  I use the variable names through my script to reference the underage and of age birth dates.  Here is an example:

The FitNesse script:

''Get underage and of age dates for 18 and 19 year olds''

!|Get Date Based On Today          |

|Add Years|Add Days|Resulting Date?|

|-18      |1       |>>UNDERAGE_18  |

|-19      |1       |>>UNDERAGE_19  |

|-18      |0       |>>OFAGE_18     |

|-19      |0       |>>OFAGE_19     |

!|Check Age of Majority|

|Province State|Birth Date   |Am I Underage?|

|MB            |<<OFAGE_18   |Yes           |

|MB            |<<UNDERAGE_18|No            |

|BC            |<<OFAGE_19   |Yes           |

|BC            |<<UNDERAGE_19|No            |

In FitNesse, the final result including the acceptance criteria above looks like this:

 (Note: The example above should probably be written as a unit test because it is fairly straightforward, but it simply illustrates how to use the date logic that I'm using as part of larger acceptance tests.)

FitNesse gets the gold!

I've been using FitNesse for just over 3 weeks and I am pleased with the value it is adding to our project even though I'm only using the basic features at this time. As a former developer I'm finding it fun to use because I have to write a little bit of code in order to create an interface between FitNesse and each service that I'm testing. Most of our testing on the project was going well and we were avoiding major errors - until last Thursday...

A change in the code from a newly completed work item resulted in 81 of our 191 tests failing. Imagine how long it would take to re-run all 191 tests manually to find out that 81 had failed. Imagine how long it would take to re-test all 191 tests manually to make sure they were fixed. As you can see from the image below, it took us 39 minutes to find, fix and re-test all 191 tests. Thanks FitNesse.

** Update 6/28:  This project had the highest quality metric that I've ever been a part of in terms of defects / month / developer.  FitNesse was a big part of that success.

Fitnesse with C#

I've decided to try out Fitnesse on my current project so I attempted to find a complete tutorial on fitnesse with C# today. The best tutorial I found is contained in the following 2 links:

1. Installing FitNesse
2. Writing your first Hello World test

This is a great tutorial. Although in the end I figured everything out, there are a few steps that were implied in the tutorial that I missed on the initial pass. Here are those steps:

a. Leave the command line open. You need to execute this command line everytime you want to use Fitnesse. If you close the command line, Fitnesse will not work.
b. When downloading the FitSharp binaries from github, place the contents of the zip file in a "dotnet2" folder that you create as a subfolder in the folder that you downloaded/installed the .jar file to.

After adding those 2 steps, everything worked great. Thanks to Gojko for all the work you put into the tutorial at the links above. I found the Fitnesse User Guide to be lacking.