Bug Management
Funny how things take a new light when you measure them. Take bugs for instance. When I started on measuring the time our current, reported, bugs have been opened I was expecting something like this:
For full disclosure, we use five level of priorities and we have a policy to fix all outstanding P1 and P2 bugs before shipping a release. What I found was this:
So effectively, we are not fixing anything that is below P2 in a reasonable amount of time. P5 are unlikely to be fixed anytime. Still we spend time managing that inventory of bugs (trying to avoid duplicate bugs, reviewing the priorities from time to time). In pure waste.
A second thing came to my mind looking at the mean time bugs stayed open. Given the amount of changes the code base undergoes in 6 months, what is the validity of a bug that was recorded at that time and was not considered important enough to be fixed ?
I will throw in a few ideas here that could lead to improvement in this area. One thing I would love is to be able to time expire defects.
Anything older than a coupe of releases should disappear automatically. Assign a high priority to defects reported from the production as they are truly visible and of value to the users who reported them. Reduce the number of priorities to three. Do not bother to record things that are evaluated below P3 in our current scale. Stop the line for anything
that we know we want to fix at the time of discovery. Each of these ideas will require a dedicated post (coming soon).
What are your thoughts on defect management ? Are these ideas outlandish ? How do you manage defects efficiently in your projects ?
February 11th, 2009 at 7:47 pm
Careful with the debt that you are accumulating if you ignore P3s and below. this might be technical debt, user experience debt, refactoring debt… but you just don’t know
I would propose at least two things:
(1) check if the users care and agree that x units of capacity will be used in each iteration to pay down some of the debt, and
(2) create tests for P3s and lower wherever possible to at least move the information from the bug logs into your code.
Assuming it is dirt-cheap to write tests, the second one means that a developer working on related code can see the associated tests when they are also cheapest to address. This makes the investment in fixing P3s as low as possible.
One related question:
- Would you be able to provide a similar graphic with three columns (P1, P2, P3-and-lower) showing number of bugs closed in the last 6 months?
César.
February 14th, 2009 at 1:36 am
No I cannot really provide the information you request.
As for the P3 and below, they are absolutely a debt of some kind. They are actually IOUs given by the development team to the users. We know that most of them will never be repaid.
In my experience, there are two efficient strategies to eliminate this debt: constant refactoring supported by tests and bundling bugs in an area with new work in the same area.
I like the idea of making the bug visible in the code or in the tests. It may be interesting to maintain a list of failing tests rather than bugs reports !