In the recent past I noticed an interesting property of self adjusting systems. They self adjust !

So when work in progress started to accumulate and stories ‘completed’ piled on the doorsteps of QA, production of new features essentially stopped. Granted, part of it was due to the number of bugs found. But given the level of energy displayed by the team, that was not the whole reason. Here is how physical systems work: when a pipe is blocked water stops flowing and typically an equilibrium is reached where water becomes stagnant at a stable level. When no equilibrium is reached it is called a flood.

Since then our team’s management has had a good idea, tell people what they were expected to finish by the end of the year. Not surprisingly, setting a clear goal energised people a bit and production has started again. Work in progress has tripled and the pile in on QA’s doorstep has doubled. Nothing short of normal in a two iteration period.

On the positive side we have the opportunity to get the development team rolling into the new year with the warm feeling of mission accomplished. Sure we now all know what mission accomplished means. It means there is a whole more work to be done before the work is actually done and delivered. But having a sense of accomplishment, given the tough economy and looming work force reduction, will be invaluable.

On the negative, let’s brace for a good month of flooding.

Joanna Zweig and César Idrovo have embarked on a journey to discover the coveted team jello, the thing that makes team jell. Best of luck to them.

Can’t wait to get a taste of it really !

I have just finished this book
this book by David J. Anderson and it was truly a breakthrough for me. It enabled me to go beyond the religious feeling of Agile to management science and why Agile methods work. Sometimes. It also lead me to finally understand what my company’s business is !

My book list just got longer by adding Domain Neutral Component, Theory of Constraint and Feature Driven Development to it.

And I encourage everyone to read the excellent CMMI-Agile article as well as another one on negotiations (both through David J. Anderson’s blog).

This talk was primarily about how and why application teams (and their architects) have a hard time engaging enterprise architecture.

I will retain the single strongest thought of the talk: enterprise architecture is a stake holder of an application among a lot other stake holders. And the second strongest thought: enterprise architects are useful to bring a transversal view of a company’s technological portfolio.

The speakers went to great length to not antagonise enterprise architects and sounded at times as if it was a PR exercise (or maybe a rehearsal for a different public ?) to convince architects that agile teams are in a better position to support their goals than other structures (by giving them more visibility into the projects and giving them a say in what gets done).

I believe that half the talk was missing though: agile projects are stake holders in the enterprise architecture. In other words it is a two way road because the enterprise architects are technological enablers and therefore dynamic teams have an interest in influencing the architectural choices and the tools used to implement them.

All too often (I have three examples in the past couple of years), enterprise architecture’s choices have been embodied by inadequate product. The architectural choices were not contested (they were actually adopted or considered as a valid constraint). The solution was left to gather dust.

So enterprise architects, let the application teams help you help them.

And I have to admit I really like Martin Fowler as a speaker.

This was such an interesting talk. At least for those of us who are interested about what drives us developing software. As a side note this came up a few times during questions after Kent Beck’s keynote but the answers were rather less interesting being mostly about money and the good of humanity. This talk echoed with a lot of ideas I have been munching on for the past year about the philosophical make of Agile developers. I also have a rather keen interest for japan.

Dave’s talk was a crash course, impressionist, education in Zen philosophy, the arts ranging from the degree of enlightenment to the prowess that the fully enlightened and what it is that the disciple seek. I am not sure what the point of the talk was though. I think it was a warning that agile in software development cannot be the recipe of the day. It is a way of life, a path to enlightenment (in the Zen sense), in other words a quest for perfection through constant improvement. Although being a software developer is not as noble as being a monk, a warrior or even a sword craftsman.

Unfortunately, being that I am rather shy and I did not go to the conference parties I could not validate my impressions with Dave West and I have the impression it would have been a very long conversation. So please, if you attended this session, correct me ! I will write on this subject further in another post.

And by the way, should we talk about transcendence or immanence ?

A week ago David J Anderson graced the Bay APLN with a rerun of his talk about Achieving Success With Agile Management.

David J. Anderson quickly departed from a lot of the Agile ideals to dive into the reasons why the Agile practices work: limit the managerial disruption, measure the process, the relationship between work in progress, lead time and quality. It was also a very good introduction to the principles of Kanban applied to software development (and now obviously I would love to try it…).

Overall successful event and an interesting talk that was for me a new step towards understanding how software development works. No doubt additional reading will further all this. My thanks to the BayAPLN for making it happen.

It is a given that agile practices require an open and trustful environment to succeed. It is also given that agile practices help promote such an environment. But in a development project it is hardly enough.

In a development team, I like to characterise the level of trust I need to be comfortable with the simple question “Would I go to war with that guy in my platoon ?”. This is obviously figurative as I have never been in a war and cannot begin to imagine the reality of it. The idea in that in a combat unit if a grenade is thrown in the middle of the unit, the soldier who sees it first will cover it with his body to protect the rest of the unit. On the same idea soldiers rotate at the head of a column where you are more likely to be shot or to trigger a trap. These are only two examples of what lies behind the question. In the more peaceful environment of an agile team it means that when someone discovers technical debt, that person (that pair) needs to address it on the spot before it becomes a problem for others in the team. It means that the riskier tasks are rotated but affect a single member of the team at a time, limiting the risk that others will be affected.

This is why, for me, the question “Would I go to war with that guy in my platoon ?” summarises the kind of ethics I have come to like and expect in my team mates. A team that shows this kind of ethics is probably “jelling”. A team that shows opposite behaviours is probably falling apart and will drag the project in its wake.

I almost forgot. The ethics are important in an agile context because the process will emanate from them, making the team truly self organised, if not self directed.

I had a discussion with a colleague the other day about the necessity of regression testing in fast release cycles.

I argued that manual regression is not necessary then, aside from the fact that it is typically in direct contradiction with the idea of having fast release cycles. Manual regression simply takes too much time for a system of significant size to allow monthly releases.

I believe firmly in automated regression. As a matter of fact we have three suites of regression test: unit tests, developer written integration tests and QA written silk tests. I also believe that the purpose of regression tests is to give the development team some confidence that no major regressions were introduced since the previous release. The word “major” in the previous sentence is very important, as is the expression “some confidence”. In other words the level of effort that goes into the regression testing is highly dependent on the appetite for risk the team has.

There are three good arguments for short release cycles and therefore limited manual regression.

First manually regressing a system is inefficient and should be limited to parts that cannot be automated (due for instance to the necessity to co-ordinate with other systems). Not automating the regression tests that can be costs development time (because feedback comes late in the process as bugs) and QA time. It obviously limits the ability to release in short cycles (our manual regressions currently take around three weeks for the entire QA team, leaving a week to test an entire month of development).

Second if the release cycle is short enough, most of the areas touched during the development cycle will have been tested as part of testing the new features. This may not be very thorough as far as regression goes but should uncover most “major” regressions and as a result improve the existing regression test suites.

Third having fast release cycles, including partially or fully automated deployment processes limits the risk created by regressions by providing an easy way to address them quickly, as they are discovered in the production system.

Should I underline that none of this is acceptable for critical systems. That is why the thoroughness of regression is dependent on the risk appetite of the system’s owners. An obvious follow-up question is “How do I know how much risk I am taking ?”.

There is usually a time in a project when the team forgets why things are done. Still for a long time we go on with the mechanics until someone questions them openly. The chicken continues to run.

It was obvious for all of us that, in their existing format, our retrospectives were useless. People would come up with well and not well items easily enough but the laundry list would never be acted upon and most of the subjects never discussed in depth. It was also clear to some of us that retrospectives have the potential to be greatly useful. Not to mention that if the retrospectives stop you might as well abandon agile altogether as evolution of the development process becomes impossible.

It so happened that a week before we tried to tackle this issue, Chris Sims facilitated a monthly meeting of the BayAPLN using group wisdom techniques. I set out to have the same applied to our retrospectives.

So for a few months now our retrospectives have followed the following process:
five minutes silence (and it is something to have silence in a meeting room) for each participant to think about something that was good or bad,
go a few times around the room to collect the notes (along with a brief explanation), one per participant and per round, highest priority should be given first, no duplicates,
once we have enough items, distribute voting stickers and get people to stand up, go to the board and vote (another nice addition to an otherwise sit down meeting),
tally the votes,
talk about the first few elected subjects,
after each subject assign an action item to one of the participants to either correct the problem or make sure good things continue to happen.

There are three important additional rules that make the process work. First each participant can put only one vote on a subject. Second the overall discussion is timed box. Third participants can ask for a conversation to be stopped (by means of a physical token like a stop or pause sign) and moved to a dedicated meeting.

The result of this is very positive as we are finally acting to continuously enhance our process. It also has an energising effect on the kick/close meeting by breaking the sit down routine.

Or How thorough QA can lead to lower quality ?

Indeed this is a paradox but it appeared very clearly to me after observing a change in the QA style in our team. Before this realisation I would have happily argued that thorough QA is the best thing that can happen to a project. The paradox unfolds in three parts.

Firstly it makes it harder (and more importantly longer) to finish stories. I understand that it also avoids accepting low quality work. However the QA perspective is slightly different (and this is good) from an actual user’s. So thorough QA, uncovers plenty of minor problems that do not significantly impact user experience or that would be minor production issues. This generates frivolous work that delays the development of more important stories, bug fixes or refactoring, thereby reducing the overall quality of the product.

Second, after a while the QA style modifies the developer’s attitude. When QA is thorough there is a sense of false confidence in the QA process that generates sub par unit and integration tests. This in turns leads to more bugs. Also, when QA is too picky before accepting a story, it removes one of the incentives to write well tested code because the difference in terms of the time it takes to close a story with good tests or not is marginally different. This effect in general is all about removing quality ownership from the development team and transfer it to the QA function.

Lastly the combined effect of the first two is that velocity falls, the project gets behind, people (well mostly managers) get nervous and start sending the message that quality is not important. The first signal is to encourage QA to log a bug but pass the story. The second signal is to de-prioritise existing bugs, effectively ignoring them (as we all know bugs underneath a given critically are never fixed and fall into the priority blackhole).

And there you have the QA paradox. Does this resonate with your own experience ?