This article is the second in the series started with the use of cost.

In agile software development the cost of a story has this interesting property that it is highly correlated to the time it takes to implement it. Consciously or unconsciously cost is used as a proxy variable for time. Actual implementation times in turn tend to be hijacked by managers to measure performance by comparing them to cost estimate. Points are often used to remove that parasitising.

While using points is good, what we really need, the purpose of estimating cost in the first place, is the ability to layout project plans in terms of time. In comes velocity that enables the transformation back from points into time. I have witnessed enough attempts to manipulate the velocity either to look good when reporting in or to transform it into a measure of productivity (including at the individual level) to have entirely lost faith in its usefulness. Burn up charts are a lot more useful. They illustrate trends rather than instant pictures, they keep people at the project level rather than the individual contributor and they provide a view of the evolution of scope.

Finally let us focus on the estimation process. I firmly believe that this process is very important as it is unveiling the real work that will be needed to complete a story. It is a crucial analysis and design process. It is very unfortunate that the cost, the number, that comes out of the process considerably overshadows the other outcome: the establishment of common knowledge in the team. It is the learning that happened that is in practice the all important outcome. This learning is sometimes expressed in artifacts, from new stories being created, to new examples being attached to the story, to a change in the definition of done. All of which are rich expressions. Cost in contrast is the poorest of them all. Why then try to estimate it or even make the focus of the process?

Estimating should not be the focus of the analysis process. The focus of the analysis process should be to ensure the stories are ready to be implemented. In other words make them of a tractable size with an agreed upon definition of done. Tractable size means they all have more or less the same cost. The cost of the story becomes a measurable random variable that provides some indication on the predictability of the process. The distribution probability of that variable can be influenced by changing the process. And in return the effect of process changes can be measured usefully. It provides management with a tool to forecast the future and to influence it. It removes the cat and mouse game that estimate versus actuals tend to become. It helps measure the performance of the project rather than that of individuals. And last but not least it focuses project teams on risk adjusted value rather than cost to prioritise their work.

That is what we have been experimenting with for the last 10 months. Although we do not actually measure the monetary cost of a story but rather its lead time. It is the cost in terms of time, an acceptable approximation considering the previous argument that the bulk of cost is personnel. So far (excluding vacation induced outliers) we average 6.23 calendar days per story from analysis start to close with a standard deviation of 3.82. Our targets for those numbers are 8 and 2.5 respectively meaning we would trade a higher average for less fluctuation. The probability distribution derived from this data also enables us to make useful predictions about the possible lateness of a story, when the lead time breaks the 3 sigma barrier, and to react to it earlier. For instance a story that reaches 8 days of lead time has 28% lateness.

Last tuesday’s meeting at the BayAPLN was a talk by Jim Highsmith entitled ‘Beyond Scope’. The talk itself was very interesting and did trigger some thoughts that I will try to organise in four or five posts. This first one is about cost.

How is cost estimated ?

There are numerous articles, methods and even books that describe how to do cost estimation for agile stories. In my limited world the consensus seem to be that cost estimation should be relative rather than absolute. It can be expressed with numbers or words. It also seems that one of the preferred methods for cost estimation is the planning poker. In that space more formal methods have been proved to work although they are rather tedious to practice.

Cost estimation for planning

Usually it is used to give an estimate of when the project will finish. As it happens, in software intensive systems, most of the operating expense is the payroll and so setting the length of the project is also essentially fixing its cost. And in many cases people will jump from one notion to the next. Now the trick is that to go from relative cost estimation to cost planning, some time equivalence has to be defined.

While we all know that circumstances (such as technical debt, absence of key team members and the colour of the sky) will make these equivalences vary over time, they are usually used in a normative way. In other words the team’s performance will be evaluated with regard to its ability to match the equivalent time that was initially chosen. In the worst cases it will be used to perform individual evaluation.

Cost as a proxy for value.

This however is a minor managerial effect. Use of cost estimation has a few implications of its own. First, while this is rarely expressed it means the scope is set. Otherwise, the planning is meaningless and what use do we have of the cost estimation ? I hear your answer: to give the business an idea of how to prioritise the stories. I was more than once on the losing side of arguing the business should prioritise before cost is known. In other words, cost is used as a proxy to value.

Is cost a good proxy for value ? Is something expensive a worst value than something cheap ? Or the contrary ? Worse, in a software system actual cost is usually dependent on the sequencing of stories. Or on how many stories a structural cost can be amortised.

To sum up.

Cost estimation is easy to come up with. It appears to be an easy control variable to measure the performance of the development process. It also appears to be an easy indicator to determine the priority of stories. In reality, is it good at any of that ?

In our process we have started using two metrics in addition to the cumulative flow diagram indicator. These metrics are tracked against target values to measure if the process is in statistical control.

The first metric is the lead time. It measures the time in days between the moment a work item enters our process and the moment it is declared finished and exists the process.

The second metric is the throughput period (the inverse of the throughput rate) which is the time in days between the end dates of successive work items. If we note two cards C1 and C2 with start dates SD1 and SD2 and end dates ED1 and ED2, then the lead time for C1 is (ED1 – SD1) in days and the cadence for that period is ED2 – ED1.

Over a period of time these measures can be plotted against a target mean (10 days of lead time and 4 days of throughput period) and a target deviation (5 days for the lead time and 2 days for the throughput period). These targets can be changed independently as our process matures. Improvement to the means benefit the delivery speed. Improvements to the deviation benefits the predictability.

Finally you will find here the model I use (for Apple’s Numbers) to visualize these process metrics.

Learning this lesson cost us two weeks of throughput. Here is what happened.

We started work on a feature that was dependent on external resources for completion. In anticipation of a quick completion (based on past experience with other providers), we filled our “Analysis Compete” queue with work that depended on the completion of the work in development.

Of course, completion was delayed and it involved a lot of waiting and very little work. So the bottleneck resource, me, was under utilized. As the delays dragged from day to day we tagged the work with an issue and thought of pushing some other work instead. But all the work in our stream was dependent on the resolution of the issue. I did not want to break WIP limits so we could not simply start a new work item. So I kept myself busy with some refactoring.

There are two bad things that stemmed from this. First we lost throughput. Second I did work that will not be verified as a story would have. The good thing to come out is that we reflected on how to improve.

The first improvement is obviously to limit dependencies between work items that are in process. In particular those for which we do not fully control the completion. The second improvement is to try to avoid saturating our queues, in other words avoid a one hundred percent utilization of them. The third improvement is that we are now comfortable dropping work in process when we find out that either it cannot be completed in a reasonable amount of time or it does not have the value we anticipated.

What do you think ? Are there other lessons that we should learn ? Do you have any advice on how to anticipate these issues ?

Backlog is not part of our process anymore. Our process starts with analysis and ends with done. This is a deliberate choice, a choice we always have to remember we made because it is so easy to go back to dumping every single idea into a card and accumulate an enormous backlog that then has to be prioritised, estimated, trimmed. We also operate without a bug list, at least so far. I have already blogged about the advantages of that.

How do we plan then ?

We use a roadmap. The shorter the horizon, the more fine grained. I use a template from enthiosys. We record functional and architectural goals on it. We record events (demos, vacations, code freezes). We also record our target markets. The roadmap has a lot of white space. Not all categories have something at every time horizons. What this means however is that we implicitly estimate when features will be completed from a very high level. This in turn gives us a lot of room to adjust what we actually deliver using dimensional planning. When we need to decide what we do next we use our roadmap and we usually make the decision in a few seconds. That is when the analysis work starts that will cut a workable story. What does not fit in the story is not recorded but will probably resurface on the next round, informed by what we learned while playing the previous story.

What about the product vision ?

We have from the beginning created a set of documents that we keep current. One is a contextual diagram that shows how this kind of product generally fits in our clients’ processes. Another one shows how it fits in the software the company produces (which is a much larger thing). Finally we have a diagram that outlines the end state of our product with a broad view of the functions we want to build. The roadmap takes over from this high level vision with a time horizon of a year or so. In addition we have an architectural context and a domain model established before we start on a functional area.

What’s the catch ?

There are several things that make this possible. First we are a very small team (one developer, one BA/QA). We both have a significant expertise in the domain and in that specific product. This process can certainly scale beyond that size but I am not sure how much. Second, we were hired to build that product and therefore we have a lot more control over how we manage things. We also do not have existing clients or contractually agreed scope to deal with.

What are the advantages ?

Primarily we removed many of the things that generate pain in a software development process. No estimates means no actuals (more on this). No backlog makes it very cheap to rearrange our priorities. No detailed long term plan means we do not have the temptation to adhere to it. Instead we reshape the plan constantly to reflect the new information we acquire. Finally all of this provides us with the material to efficiently communicate with our prospects and clients. We show enough structure that they easily understand where we are going and at the same time we have enough flexibility to incorporate their feedback in our plan. And the same thing applies to communication with upper management.

That is a short presentation of how we (so far happily) operate. What is your experience like ? Have you tried similar things ? Or other things ? How did they work out ?

It has been two months since we set on the Kanban journey. It is time to make a few observations.

First it is very hard to fight the urge of piling more work in our queues. We have too many ideas, many of them looking good enough that we would like to record them. It is a conscious effort to resist this and remind ourselves that, if these ideas are good they will either resurface coming from stake holders or we will remember them. If not, chances are we would not find them in a laundry list anyway.

Second it was enlightening to notice how small stories and short cycle time is energising. Short cycle times give an incredible impression of control over the process. They are also morale boosters as progress is continuous. Now this contrasts significantly with a longer story we played recently, which incidentally could have been broken down. That story gave the feeling of a squid, a temporary loss of control. Your vision becomes a tunnel, you lose peripheral vision and you hope that this will all end well. In the future we will proactively split stories and we will have to resolve the challenge of possibly breaking our WIP limit.

Third we got to experience how cycle and lead times are impacted when the bottleneck (me sadly) is loaded with additional work. We have this inconvenient position where we should reuse code that is in no shape to be reused and therefore needs to be cautiously refactored. That work cannot progress at the speed we move so we have to somewhat duplicate it while we work towards a good solution. Last month I had a green light to make one of the changes and did it. But this is additional work that has increased the lead time and cycle time without direct value to the project. In effect the project is working pro-bono for the rest of the firm. The positive about this is that the effect is immediately visible. In the future we will track this kind of work as issues.

Third, my coworker and subject matter expert was enlightened by Chris Matts’ cartoons. He did not use to understand how Agile in general and our own process in particular can be so efficient in the financial software industry. The Agile 2009 cartoon was an important stepping stone for his “conversion”.

How do you record ideas without piling them ? Do you have advice on how to handle external work ? How do you handle back-flow into limited size queues ?

A very stimulating talk again yesterday at the BayAPLN meeting by Joshua Kerievsky. The talk was about “The Art of Choosing an Agile Transition Style” and was based on an analogy between painting and Agile Transitions.

I am not sure I fully buy into the analogy of painting style to agile transition style. I actually feel the painting style is more useful as an analogy for the practice style (rather than transition only). And that is what was actually talked about most of the time.

A specific point sticked out during the talk and in the questions for which no good answer was articulated. It stemmed from the remark that the creators and masters of new styles (impressionism, cubism, expressionism) had a classic (baroque) training and were perfectly able to paint in that style. And so the question came of how much knowledge do people need to know about the pre-existing styles before being able to apply more modern style.

I think the question is best answered by making a distinction between technique and style. The impressionists were all trained in the techniques that supported the baroque style. And this knowledge enabled them to create new techniques, experiment with them and then give rise to new styles. In this case the style emerges from the techniques that are applied to the art. Obviously, once the technique is created it can be taught and the student of that technique can then paint in the new styles. The deeper their technical knowledge the more styles they have available to express their art or satisfy their patrons.

And the lesson I take out of this is that to apply agile in various environments one has to be well versed in all the techniques that exist to successfully select the ones that will support the desired style. It seems to indicate also that to successfully transition from one style to another one needs a good understanding of the techniques and aspirations that created the old style in addition to knowing the techniques that will support the new style.

Stimulating is it not ?

I have heard and participated in a lot of conversations recently about tracking the time actually taken to complete a card. This is usually imposed to the team by management. I have never seen a team interested in how much time they actually spent on a work item. What are the reasons to track actuals ?

The first reason is to track how people spend their time. The idea behind it is usually that you cannot trust people to work hard. And that may be true in some contexts. However, the same people who distrust people’s work ethics entrust them with gathering the data. And so they essentially get the ideal values randomly adjusted so that they look real. While this is a useful practice at a personal level (see the pomodoro technique for instance), at a team level it really comes from the ‘look busy’ frame of mind. Throughput should be the thing management cares about.

The second reason is to try to get better at estimating. So managers want to try actuals and compare them to estimates. And then if the actuals do not match the estimates put pressure on the team to get better estimates. So for those of you who believe in that, here is what happens. Developers will start by padding their estimate so that they are pretty sure the actuals won’t exceed the estimate. Second, remember you hired them because they are smart, so they will never finish early to avoid the suspicion of padding. So you will get perfect detailed estimates and still the project will be late. And if, being clever yourself, you cut the estimates and people do not have time to finish within the estimate, then they will cut corners, drop quality and finish within the estimates anyway. And the project will be late because of those bugs that need to be fixed. By the way did you know that the best estimation is usually at a coarse grain level because then errors tend to offset each other ?

The third less harmful reason for tracking actuals is to improve the time forecast. This is essentially a statistical use by we try to measure the relationship between ideal days (or points, or size) and actual days. Even with this innocuous goal it is a waste of time. First the information is already available in the burn up (or down) chart. Second it tends to be wrong because early in a project there is not enough data to make it statistically relevant and later in a project chances are circumstances have changed (personnel, techniques, technology may have changed, technical debt may becoming a burden etc).

So why waste time and moral on actuals ? Even better, once you abandon actuals, precise estimates become irrelevant and you can get away with relative sizes. And relative sizes are very quick and easy to get consensus on and you therefore save time that too. So please abandon the collection of actual times !

Inspired by the work of the Limited WIP Society and kanban community, I set on my journey to create my very own kanban process for my team of me and my subject matter expert/product manager/QA.

First we identified the stages and queues of our development process. We decided that the process starts with the Analysis of a work item. To be quite honest some informal analysis and some preparation work to clear out dependencies takes place before that but we did not formally include it in our process. Most of it is tracked at the road map level. The analysis yields one card, trimmed to a manageable size with the acceptance criteria expressed as a few BDD-style scenarii.

Then there is an Analysis Complete queue. It buffers developers.

Then there is the development in progress stage where cards are again actively worked on. We try to follow best practices for the most part. A card cannot leave this stage until the tests pass, the reports are fine (no cyclic dependencies, no significant PMD or FindBugs warnings, enough coverage) and the acceptance scenarii have been automated with cucumber (and the awesome cuke4duke maven plugin).

Then there is a development complete queue. It buffers QA.

Then there is the QA in progress stage. This is about verifying the scenarii manually and verifying their automation. Any issue or concern raised is immediately addressed. If the issue needs tracking, an issue card is created and attached to the card.

Then there is the closed stage which is the exit of our process.

We do not track statistics internal to the process aside from the cumulative flow diagram. The administrative cost of that seems too high for such a small team. We do track the cycle time of each card in our process.

I will write more on the observations from the first month in an upcoming post.

Some organisations, or parts thereof, have a development process that lives squarely in chaos. Worse, many members of those organisations believe it is a normal, sometimes even desirable, state. The others have left.

What I have noticed in those environments is that new techniques and technologies do not get adopted. Instead a passive-agressive attitude exists, whereby the change is not supported and when things degrade people are quick to use it as proof their old familiar ways are superior. Any attempt to introduce any kind of methodology that is “not invented here” is doomed in this context. I am quite convinced that any change is actually doomed. I may get the opportunity to observe this in the introduction of a “waterfall” process in my current company.

Worse however, while many would think that an agile methodology is more appropriate for those organisations (and they are probably right), it is almost impossible to successfully introduce it. In most cases members of the organisation lack the practice of self discipline necessary to support an agile process. And building that requires in turn a full cultural change where, initially at least, the disciplined application of good practices is valued more than the result.

How would you go about it if you are not the CEO ? Do you have gardener’s tip to successfully grow agile seeds ?