Backlog is not part of our process anymore. Our process starts with analysis and ends with done. This is a deliberate choice, a choice we always have to remember we made because it is so easy to go back to dumping every single idea into a card and accumulate an enormous backlog that then has to be prioritised, estimated, trimmed. We also operate without a bug list, at least so far. I have already blogged about the advantages of that.

How do we plan then ?

We use a roadmap. The shorter the horizon, the more fine grained. I use a template from enthiosys. We record functional and architectural goals on it. We record events (demos, vacations, code freezes). We also record our target markets. The roadmap has a lot of white space. Not all categories have something at every time horizons. What this means however is that we implicitly estimate when features will be completed from a very high level. This in turn gives us a lot of room to adjust what we actually deliver using dimensional planning. When we need to decide what we do next we use our roadmap and we usually make the decision in a few seconds. That is when the analysis work starts that will cut a workable story. What does not fit in the story is not recorded but will probably resurface on the next round, informed by what we learned while playing the previous story.

What about the product vision ?

We have from the beginning created a set of documents that we keep current. One is a contextual diagram that shows how this kind of product generally fits in our clients’ processes. Another one shows how it fits in the software the company produces (which is a much larger thing). Finally we have a diagram that outlines the end state of our product with a broad view of the functions we want to build. The roadmap takes over from this high level vision with a time horizon of a year or so. In addition we have an architectural context and a domain model established before we start on a functional area.

What’s the catch ?

There are several things that make this possible. First we are a very small team (one developer, one BA/QA). We both have a significant expertise in the domain and in that specific product. This process can certainly scale beyond that size but I am not sure how much. Second, we were hired to build that product and therefore we have a lot more control over how we manage things. We also do not have existing clients or contractually agreed scope to deal with.

What are the advantages ?

Primarily we removed many of the things that generate pain in a software development process. No estimates means no actuals (more on this). No backlog makes it very cheap to rearrange our priorities. No detailed long term plan means we do not have the temptation to adhere to it. Instead we reshape the plan constantly to reflect the new information we acquire. Finally all of this provides us with the material to efficiently communicate with our prospects and clients. We show enough structure that they easily understand where we are going and at the same time we have enough flexibility to incorporate their feedback in our plan. And the same thing applies to communication with upper management.

That is a short presentation of how we (so far happily) operate. What is your experience like ? Have you tried similar things ? Or other things ? How did they work out ?

It has been two months since we set on the Kanban journey. It is time to make a few observations.

First it is very hard to fight the urge of piling more work in our queues. We have too many ideas, many of them looking good enough that we would like to record them. It is a conscious effort to resist this and remind ourselves that, if these ideas are good they will either resurface coming from stake holders or we will remember them. If not, chances are we would not find them in a laundry list anyway.

Second it was enlightening to notice how small stories and short cycle time is energising. Short cycle times give an incredible impression of control over the process. They are also morale boosters as progress is continuous. Now this contrasts significantly with a longer story we played recently, which incidentally could have been broken down. That story gave the feeling of a squid, a temporary loss of control. Your vision becomes a tunnel, you lose peripheral vision and you hope that this will all end well. In the future we will proactively split stories and we will have to resolve the challenge of possibly breaking our WIP limit.

Third we got to experience how cycle and lead times are impacted when the bottleneck (me sadly) is loaded with additional work. We have this inconvenient position where we should reuse code that is in no shape to be reused and therefore needs to be cautiously refactored. That work cannot progress at the speed we move so we have to somewhat duplicate it while we work towards a good solution. Last month I had a green light to make one of the changes and did it. But this is additional work that has increased the lead time and cycle time without direct value to the project. In effect the project is working pro-bono for the rest of the firm. The positive about this is that the effect is immediately visible. In the future we will track this kind of work as issues.

Third, my coworker and subject matter expert was enlightened by Chris Matts’ cartoons. He did not use to understand how Agile in general and our own process in particular can be so efficient in the financial software industry. The Agile 2009 cartoon was an important stepping stone for his “conversion”.

How do you record ideas without piling them ? Do you have advice on how to handle external work ? How do you handle back-flow into limited size queues ?

A very stimulating talk again yesterday at the BayAPLN meeting by Joshua Kerievsky. The talk was about “The Art of Choosing an Agile Transition Style” and was based on an analogy between painting and Agile Transitions.

I am not sure I fully buy into the analogy of painting style to agile transition style. I actually feel the painting style is more useful as an analogy for the practice style (rather than transition only). And that is what was actually talked about most of the time.

A specific point sticked out during the talk and in the questions for which no good answer was articulated. It stemmed from the remark that the creators and masters of new styles (impressionism, cubism, expressionism) had a classic (baroque) training and were perfectly able to paint in that style. And so the question came of how much knowledge do people need to know about the pre-existing styles before being able to apply more modern style.

I think the question is best answered by making a distinction between technique and style. The impressionists were all trained in the techniques that supported the baroque style. And this knowledge enabled them to create new techniques, experiment with them and then give rise to new styles. In this case the style emerges from the techniques that are applied to the art. Obviously, once the technique is created it can be taught and the student of that technique can then paint in the new styles. The deeper their technical knowledge the more styles they have available to express their art or satisfy their patrons.

And the lesson I take out of this is that to apply agile in various environments one has to be well versed in all the techniques that exist to successfully select the ones that will support the desired style. It seems to indicate also that to successfully transition from one style to another one needs a good understanding of the techniques and aspirations that created the old style in addition to knowing the techniques that will support the new style.

Stimulating is it not ?

I have heard and participated in a lot of conversations recently about tracking the time actually taken to complete a card. This is usually imposed to the team by management. I have never seen a team interested in how much time they actually spent on a work item. What are the reasons to track actuals ?

The first reason is to track how people spend their time. The idea behind it is usually that you cannot trust people to work hard. And that may be true in some contexts. However, the same people who distrust people’s work ethics entrust them with gathering the data. And so they essentially get the ideal values randomly adjusted so that they look real. While this is a useful practice at a personal level (see the pomodoro technique for instance), at a team level it really comes from the ‘look busy’ frame of mind. Throughput should be the thing management cares about.

The second reason is to try to get better at estimating. So managers want to try actuals and compare them to estimates. And then if the actuals do not match the estimates put pressure on the team to get better estimates. So for those of you who believe in that, here is what happens. Developers will start by padding their estimate so that they are pretty sure the actuals won’t exceed the estimate. Second, remember you hired them because they are smart, so they will never finish early to avoid the suspicion of padding. So you will get perfect detailed estimates and still the project will be late. And if, being clever yourself, you cut the estimates and people do not have time to finish within the estimate, then they will cut corners, drop quality and finish within the estimates anyway. And the project will be late because of those bugs that need to be fixed. By the way did you know that the best estimation is usually at a coarse grain level because then errors tend to offset each other ?

The third less harmful reason for tracking actuals is to improve the time forecast. This is essentially a statistical use by we try to measure the relationship between ideal days (or points, or size) and actual days. Even with this innocuous goal it is a waste of time. First the information is already available in the burn up (or down) chart. Second it tends to be wrong because early in a project there is not enough data to make it statistically relevant and later in a project chances are circumstances have changed (personnel, techniques, technology may have changed, technical debt may becoming a burden etc).

So why waste time and moral on actuals ? Even better, once you abandon actuals, precise estimates become irrelevant and you can get away with relative sizes. And relative sizes are very quick and easy to get consensus on and you therefore save time that too. So please abandon the collection of actual times !

Inspired by the work of the Limited WIP Society and kanban community, I set on my journey to create my very own kanban process for my team of me and my subject matter expert/product manager/QA.

First we identified the stages and queues of our development process. We decided that the process starts with the Analysis of a work item. To be quite honest some informal analysis and some preparation work to clear out dependencies takes place before that but we did not formally include it in our process. Most of it is tracked at the road map level. The analysis yields one card, trimmed to a manageable size with the acceptance criteria expressed as a few BDD-style scenarii.

Then there is an Analysis Complete queue. It buffers developers.

Then there is the development in progress stage where cards are again actively worked on. We try to follow best practices for the most part. A card cannot leave this stage until the tests pass, the reports are fine (no cyclic dependencies, no significant PMD or FindBugs warnings, enough coverage) and the acceptance scenarii have been automated with cucumber (and the awesome cuke4duke maven plugin).

Then there is a development complete queue. It buffers QA.

Then there is the QA in progress stage. This is about verifying the scenarii manually and verifying their automation. Any issue or concern raised is immediately addressed. If the issue needs tracking, an issue card is created and attached to the card.

Then there is the closed stage which is the exit of our process.

We do not track statistics internal to the process aside from the cumulative flow diagram. The administrative cost of that seems too high for such a small team. We do track the cycle time of each card in our process.

I will write more on the observations from the first month in an upcoming post.

Some organisations, or parts thereof, have a development process that lives squarely in chaos. Worse, many members of those organisations believe it is a normal, sometimes even desirable, state. The others have left.

What I have noticed in those environments is that new techniques and technologies do not get adopted. Instead a passive-agressive attitude exists, whereby the change is not supported and when things degrade people are quick to use it as proof their old familiar ways are superior. Any attempt to introduce any kind of methodology that is “not invented here” is doomed in this context. I am quite convinced that any change is actually doomed. I may get the opportunity to observe this in the introduction of a “waterfall” process in my current company.

Worse however, while many would think that an agile methodology is more appropriate for those organisations (and they are probably right), it is almost impossible to successfully introduce it. In most cases members of the organisation lack the practice of self discipline necessary to support an agile process. And building that requires in turn a full cultural change where, initially at least, the disciplined application of good practices is valued more than the result.

How would you go about it if you are not the CEO ? Do you have gardener’s tip to successfully grow agile seeds ?

César has been a good friend and a great mentor for me. He is resourceful, dynamic and energetic. As it happens he has also contributed ideas and comments to this blog.

If you do have a vote in the 2009 Agile Alliance board election, please take a moment to review César’s position statement and then vote, for him preferably !

Last week the Bay APLN had a great meeting where Chris Sims offered some of his agile learning games. The games offered the opportunity to feel and illustrate what has long been published (and is still thoroughly ignored by a large chunk of managers and workers). The first game we played was to illustrate the effects of multitasking. I have come to realise two things at the end of the game.

First what we call multitasking really is an interruption driven work process. That process is a consequence of our overall work organisation. Instead of having cadence, we usually expedite. THe way we expedite is also grounded in the idea that we should not be idle and we should be “utilised’ all the time.

Second the downside of multitasking really is the time it takes to resume a task after an interruption. It is a set up time or a transaction cost.

A couple of things were illustrated. I offered that the pomodoro technique was a great help to handle multitasking. Essentially the pomodoro technique provides cadence and thereby suppresses the interruption driven work style. In exchange it provides predictable windows to address urgent requests. In this way it forces triage between urgent and not urgent while preserving the ability to adapt the work stream. It also enforces the idea that it is good to have idle time. Other signalling techniques were offered to prevent interruptions.

It was interesting to witness that the default attitude we had to resolve the problem of interruptions was to increase the batch sizes to avoid them. Essentially the same idea as reducing the set-up time in the industry. Importantly the discussion turned to techniques to reduce the transaction cost (context switching time) associated to multiple tasks. This is the realm of getting things done techniques (to-do lists to boot). One interesting remark from David Chilcott is to note down the next step in your current task before switching to another task.

To summarise, multitasking is good. Interruption is evil.

What techniques do you use to cope with interruptions ? What do you do to facilitate context switching ?

Today I am starting a new job. Different company, different title, similar role. So I thought I would try to describe why I am moving on.

I am going back to a software vendor I left it a few years ago. I left because I was missing direct contact with the business users. I left because I felt I had reached the limits of my influence and growth there. I am going back because I am given a great opportunity both in terms of career and personal development. I have a chance to further the changes I helped introduce back then. I get a chance to apply innovative solutions. That is the bottom line for me. In my previous role, despite all the great people I met and worked with I was not getting to apply innovative solutions. And therefore I was not learning as much as I would have liked. This is the deep reason why I felt dissatisfied. The other elements that took part in my decision were only the back breaking straws.

In fact I strongly believe turnover is beneficial for companies and employees alike in the software engineering world. As long as the turnover aligns with project duration it is not overly destabilising. The company gets to hire new people with fresh ideas. The employee gets to learn new things. I wonder if knowledge workers could be classified in the same way as open space participants: pigs, bees and butterflies.

What do you think ? What kind of worker are you ?

I have already mentioned that, in my opinion, configuration is often the result of a lack of design. In other words we introduce configuration as a substitute for a design decision. There are some cases however when it is desirable to make things configurable. For example, when a software vendor needs to enable clients to plug in custom behaviour without a restart of their system.

I would contend this is a rare scenario and that in most cases injection of dependency addresses the actual need (think assembly with Spring for instance).

I imagine cases when that is not enough and I was wondering at what point does the configuration parameters start to form a DSL. I was also wondering when it becomes more efficient to create a DSL to program the adaptation to local conditions. An initial guideline could be that it is as soon as you thing of storing the configuration in an XML document.

What is your take ? Does it help to rethink configuration as a DSL ? What is your experience ?