Here is why you should use compareTo to find out whether two BigDecimals represent the same number or not.

import java.math.BigDecimal;

import junit.framework.TestCase;

public final class TestBigDecimal extends TestCase {
    public void testBigDecimalZeroEqualsBigDecimalValueOfZero() {
        assertTrue(BigDecimal.ZERO.equals(BigDecimal.valueOf(0)));
    }

    public void testBigDecimalZeroNotEqualsBigDecimalValueOfZeroDot() {
        assertFalse(BigDecimal.ZERO.equals(BigDecimal.valueOf(0.)));
    }

    public void testBigDecimalZeroEqualsBigDecimalValueOfZeroString() {
        assertTrue(BigDecimal.ZERO.equals(new BigDecimal("0")));
    }

    public void testBigDecimalZeroEqualsBigDecimalValueOfZeroDotString() {
        assertTrue(BigDecimal.ZERO.equals(new BigDecimal("0.")));
    }

    public void testBigDecimalZeroNotEqualsBigDecimalValueOfZeroDotZeroString() {
        assertFalse(BigDecimal.ZERO.equals(new BigDecimal("0.0")));
    }

    public void testBigDecimalZeroComparesToBigDecimalValueOfZeroDotAsZero() {
        assertEquals(0, BigDecimal.ZERO.compareTo(BigDecimal.valueOf(0.)));
    }

    public void testBigDecimalZeroComparesToBigDecimalValueOfZeroDotZeroStringAsZero() {
        assertEquals(0, BigDecimal.ZERO.compareTo(new BigDecimal("0.0")));
    }
}

I used to like overloading methods to provide a default behaviour for any object and a specialised one for some select types. That’s possible when the language supports multiple dispatch. It is not possible in Java:

import junit.framework.TestCase;

public class TestOverloading extends TestCase {
    public static final class OverloadedClass {
        public String thisIsA(final Object o) {
            return "Object";
        }

        public String thisIsA(final String s) {
            return "String";
        }
    }

    public void testIsCallingObjectFromObject() {
        final OverloadedClass tested = new OverloadedClass();
        final Object value = new Object();

        assertEquals("Object", tested.thisIsA(value));
    }

    public void testIsCallingObjectFromString() {
        final OverloadedClass tested = new OverloadedClass();
        final Object value = new String();

        assertEquals("Object", tested.thisIsA(value));
    }

    public void testIsCallingStringFromString() {
        final OverloadedClass tested = new OverloadedClass();
        final String value = new String();

        assertEquals("String", tested.thisIsA(value));
    }
}

Last tuesday’s meeting at the BayAPLN was a talk by Jim Highsmith entitled ‘Beyond Scope’. The talk itself was very interesting and did trigger some thoughts that I will try to organise in four or five posts. This first one is about cost.

How is cost estimated ?

There are numerous articles, methods and even books that describe how to do cost estimation for agile stories. In my limited world the consensus seem to be that cost estimation should be relative rather than absolute. It can be expressed with numbers or words. It also seems that one of the preferred methods for cost estimation is the planning poker. In that space more formal methods have been proved to work although they are rather tedious to practice.

Cost estimation for planning

Usually it is used to give an estimate of when the project will finish. As it happens, in software intensive systems, most of the operating expense is the payroll and so setting the length of the project is also essentially fixing its cost. And in many cases people will jump from one notion to the next. Now the trick is that to go from relative cost estimation to cost planning, some time equivalence has to be defined.

While we all know that circumstances (such as technical debt, absence of key team members and the colour of the sky) will make these equivalences vary over time, they are usually used in a normative way. In other words the team’s performance will be evaluated with regard to its ability to match the equivalent time that was initially chosen. In the worst cases it will be used to perform individual evaluation.

Cost as a proxy for value.

This however is a minor managerial effect. Use of cost estimation has a few implications of its own. First, while this is rarely expressed it means the scope is set. Otherwise, the planning is meaningless and what use do we have of the cost estimation ? I hear your answer: to give the business an idea of how to prioritise the stories. I was more than once on the losing side of arguing the business should prioritise before cost is known. In other words, cost is used as a proxy to value.

Is cost a good proxy for value ? Is something expensive a worst value than something cheap ? Or the contrary ? Worse, in a software system actual cost is usually dependent on the sequencing of stories. Or on how many stories a structural cost can be amortised.

To sum up.

Cost estimation is easy to come up with. It appears to be an easy control variable to measure the performance of the development process. It also appears to be an easy indicator to determine the priority of stories. In reality, is it good at any of that ?

Over the past few months I have grown to like the ‘given a’ syntax more and more (described here). So when I recently reactivated my Ruby On rails home project (see brikeyard deployed on heroku), I thought I would add that syntax to my rspec examples rather than use before(:each).

I have not spent too much time thinking about where to put this code. Nor did I spend much time peeling away the nested modules inside rspec. Ideas to make this better are very welcomed. I would also love to collect some feedback on this before I make a fool of myself by proposing its inclusion in rspec proper. Thanks in advance for your help !

Here is how the setup looks like in one of the specs:

describe :something do
    given {
      a StatisticalControl
      a WorkItem, {:lead_time => 1}
      another WorkItem, {:lead_time => 3}
      a ProcessStage, {:work_in_process => [a_work_item, another_work_item]}
      a ControlChart
      @project = Project.new
      @project.stub!(:lead_time_control).and_return(a_statistical_control)
      @project.stub!(:closed_stage).and_return(a_process_stage)
    }

    it "should do something useful" do
		(...)
    end
end

And here is what I added to my spec_helper.rb file:

def a(type, stubs ={})
  stub = mock_model(type, stubs)
  member_name = type.name.gsub(/[A-Z]/) {|match|
    "_#{match.downcase}"
  }
  instance_eval("
        @a#{member_name} = stub
          def a#{member_name}
            return @a#{member_name}
          end
      ")
end

def another(type, stubs ={})
  stub = mock_model(type, stubs)
  member_name = type.name.gsub(/[A-Z]/) {|match|
    "_#{match.downcase}"
  }
  instance_eval("
        @another#{member_name} = stub
          def another#{member_name}
            return @another#{member_name}
          end
      ")
end

class Spec::Rails::Example::FunctionalExampleGroup
  class << self
    def given &block
      before(:each, &block)
    end
  end
end

class Spec::Rails::Example::ModelExampleGroup
  class << self
    def given &block
      before(:each, &block)
    end
  end
end

Please let me know what you think. I welcome suggestions to improve.

In our process we have started using two metrics in addition to the cumulative flow diagram indicator. These metrics are tracked against target values to measure if the process is in statistical control.

The first metric is the lead time. It measures the time in days between the moment a work item enters our process and the moment it is declared finished and exists the process.

The second metric is the throughput period (the inverse of the throughput rate) which is the time in days between the end dates of successive work items. If we note two cards C1 and C2 with start dates SD1 and SD2 and end dates ED1 and ED2, then the lead time for C1 is (ED1 – SD1) in days and the cadence for that period is ED2 – ED1.

Over a period of time these measures can be plotted against a target mean (10 days of lead time and 4 days of throughput period) and a target deviation (5 days for the lead time and 2 days for the throughput period). These targets can be changed independently as our process matures. Improvement to the means benefit the delivery speed. Improvements to the deviation benefits the predictability.

Finally you will find here the model I use (for Apple’s Numbers) to visualize these process metrics.

Learning this lesson cost us two weeks of throughput. Here is what happened.

We started work on a feature that was dependent on external resources for completion. In anticipation of a quick completion (based on past experience with other providers), we filled our “Analysis Compete” queue with work that depended on the completion of the work in development.

Of course, completion was delayed and it involved a lot of waiting and very little work. So the bottleneck resource, me, was under utilized. As the delays dragged from day to day we tagged the work with an issue and thought of pushing some other work instead. But all the work in our stream was dependent on the resolution of the issue. I did not want to break WIP limits so we could not simply start a new work item. So I kept myself busy with some refactoring.

There are two bad things that stemmed from this. First we lost throughput. Second I did work that will not be verified as a story would have. The good thing to come out is that we reflected on how to improve.

The first improvement is obviously to limit dependencies between work items that are in process. In particular those for which we do not fully control the completion. The second improvement is to try to avoid saturating our queues, in other words avoid a one hundred percent utilization of them. The third improvement is that we are now comfortable dropping work in process when we find out that either it cannot be completed in a reasonable amount of time or it does not have the value we anticipated.

What do you think ? Are there other lessons that we should learn ? Do you have any advice on how to anticipate these issues ?

Backlog is not part of our process anymore. Our process starts with analysis and ends with done. This is a deliberate choice, a choice we always have to remember we made because it is so easy to go back to dumping every single idea into a card and accumulate an enormous backlog that then has to be prioritised, estimated, trimmed. We also operate without a bug list, at least so far. I have already blogged about the advantages of that.

How do we plan then ?

We use a roadmap. The shorter the horizon, the more fine grained. I use a template from enthiosys. We record functional and architectural goals on it. We record events (demos, vacations, code freezes). We also record our target markets. The roadmap has a lot of white space. Not all categories have something at every time horizons. What this means however is that we implicitly estimate when features will be completed from a very high level. This in turn gives us a lot of room to adjust what we actually deliver using dimensional planning. When we need to decide what we do next we use our roadmap and we usually make the decision in a few seconds. That is when the analysis work starts that will cut a workable story. What does not fit in the story is not recorded but will probably resurface on the next round, informed by what we learned while playing the previous story.

What about the product vision ?

We have from the beginning created a set of documents that we keep current. One is a contextual diagram that shows how this kind of product generally fits in our clients’ processes. Another one shows how it fits in the software the company produces (which is a much larger thing). Finally we have a diagram that outlines the end state of our product with a broad view of the functions we want to build. The roadmap takes over from this high level vision with a time horizon of a year or so. In addition we have an architectural context and a domain model established before we start on a functional area.

What’s the catch ?

There are several things that make this possible. First we are a very small team (one developer, one BA/QA). We both have a significant expertise in the domain and in that specific product. This process can certainly scale beyond that size but I am not sure how much. Second, we were hired to build that product and therefore we have a lot more control over how we manage things. We also do not have existing clients or contractually agreed scope to deal with.

What are the advantages ?

Primarily we removed many of the things that generate pain in a software development process. No estimates means no actuals (more on this). No backlog makes it very cheap to rearrange our priorities. No detailed long term plan means we do not have the temptation to adhere to it. Instead we reshape the plan constantly to reflect the new information we acquire. Finally all of this provides us with the material to efficiently communicate with our prospects and clients. We show enough structure that they easily understand where we are going and at the same time we have enough flexibility to incorporate their feedback in our plan. And the same thing applies to communication with upper management.

That is a short presentation of how we (so far happily) operate. What is your experience like ? Have you tried similar things ? Or other things ? How did they work out ?

It has been two months since we set on the Kanban journey. It is time to make a few observations.

First it is very hard to fight the urge of piling more work in our queues. We have too many ideas, many of them looking good enough that we would like to record them. It is a conscious effort to resist this and remind ourselves that, if these ideas are good they will either resurface coming from stake holders or we will remember them. If not, chances are we would not find them in a laundry list anyway.

Second it was enlightening to notice how small stories and short cycle time is energising. Short cycle times give an incredible impression of control over the process. They are also morale boosters as progress is continuous. Now this contrasts significantly with a longer story we played recently, which incidentally could have been broken down. That story gave the feeling of a squid, a temporary loss of control. Your vision becomes a tunnel, you lose peripheral vision and you hope that this will all end well. In the future we will proactively split stories and we will have to resolve the challenge of possibly breaking our WIP limit.

Third we got to experience how cycle and lead times are impacted when the bottleneck (me sadly) is loaded with additional work. We have this inconvenient position where we should reuse code that is in no shape to be reused and therefore needs to be cautiously refactored. That work cannot progress at the speed we move so we have to somewhat duplicate it while we work towards a good solution. Last month I had a green light to make one of the changes and did it. But this is additional work that has increased the lead time and cycle time without direct value to the project. In effect the project is working pro-bono for the rest of the firm. The positive about this is that the effect is immediately visible. In the future we will track this kind of work as issues.

Third, my coworker and subject matter expert was enlightened by Chris Matts’ cartoons. He did not use to understand how Agile in general and our own process in particular can be so efficient in the financial software industry. The Agile 2009 cartoon was an important stepping stone for his “conversion”.

How do you record ideas without piling them ? Do you have advice on how to handle external work ? How do you handle back-flow into limited size queues ?

A very stimulating talk again yesterday at the BayAPLN meeting by Joshua Kerievsky. The talk was about “The Art of Choosing an Agile Transition Style” and was based on an analogy between painting and Agile Transitions.

I am not sure I fully buy into the analogy of painting style to agile transition style. I actually feel the painting style is more useful as an analogy for the practice style (rather than transition only). And that is what was actually talked about most of the time.

A specific point sticked out during the talk and in the questions for which no good answer was articulated. It stemmed from the remark that the creators and masters of new styles (impressionism, cubism, expressionism) had a classic (baroque) training and were perfectly able to paint in that style. And so the question came of how much knowledge do people need to know about the pre-existing styles before being able to apply more modern style.

I think the question is best answered by making a distinction between technique and style. The impressionists were all trained in the techniques that supported the baroque style. And this knowledge enabled them to create new techniques, experiment with them and then give rise to new styles. In this case the style emerges from the techniques that are applied to the art. Obviously, once the technique is created it can be taught and the student of that technique can then paint in the new styles. The deeper their technical knowledge the more styles they have available to express their art or satisfy their patrons.

And the lesson I take out of this is that to apply agile in various environments one has to be well versed in all the techniques that exist to successfully select the ones that will support the desired style. It seems to indicate also that to successfully transition from one style to another one needs a good understanding of the techniques and aspirations that created the old style in addition to knowing the techniques that will support the new style.

Stimulating is it not ?

I have heard and participated in a lot of conversations recently about tracking the time actually taken to complete a card. This is usually imposed to the team by management. I have never seen a team interested in how much time they actually spent on a work item. What are the reasons to track actuals ?

The first reason is to track how people spend their time. The idea behind it is usually that you cannot trust people to work hard. And that may be true in some contexts. However, the same people who distrust people’s work ethics entrust them with gathering the data. And so they essentially get the ideal values randomly adjusted so that they look real. While this is a useful practice at a personal level (see the pomodoro technique for instance), at a team level it really comes from the ‘look busy’ frame of mind. Throughput should be the thing management cares about.

The second reason is to try to get better at estimating. So managers want to try actuals and compare them to estimates. And then if the actuals do not match the estimates put pressure on the team to get better estimates. So for those of you who believe in that, here is what happens. Developers will start by padding their estimate so that they are pretty sure the actuals won’t exceed the estimate. Second, remember you hired them because they are smart, so they will never finish early to avoid the suspicion of padding. So you will get perfect detailed estimates and still the project will be late. And if, being clever yourself, you cut the estimates and people do not have time to finish within the estimate, then they will cut corners, drop quality and finish within the estimates anyway. And the project will be late because of those bugs that need to be fixed. By the way did you know that the best estimation is usually at a coarse grain level because then errors tend to offset each other ?

The third less harmful reason for tracking actuals is to improve the time forecast. This is essentially a statistical use by we try to measure the relationship between ideal days (or points, or size) and actual days. Even with this innocuous goal it is a waste of time. First the information is already available in the burn up (or down) chart. Second it tends to be wrong because early in a project there is not enough data to make it statistically relevant and later in a project chances are circumstances have changed (personnel, techniques, technology may have changed, technical debt may becoming a burden etc).

So why waste time and moral on actuals ? Even better, once you abandon actuals, precise estimates become irrelevant and you can get away with relative sizes. And relative sizes are very quick and easy to get consensus on and you therefore save time that too. So please abandon the collection of actual times !