A week ago David J Anderson graced the Bay APLN with a rerun of his talk about Achieving Success With Agile Management.

David J. Anderson quickly departed from a lot of the Agile ideals to dive into the reasons why the Agile practices work: limit the managerial disruption, measure the process, the relationship between work in progress, lead time and quality. It was also a very good introduction to the principles of Kanban applied to software development (and now obviously I would love to try it…).

Overall successful event and an interesting talk that was for me a new step towards understanding how software development works. No doubt additional reading will further all this. My thanks to the BayAPLN for making it happen.

It is a given that agile practices require an open and trustful environment to succeed. It is also given that agile practices help promote such an environment. But in a development project it is hardly enough.

In a development team, I like to characterise the level of trust I need to be comfortable with the simple question “Would I go to war with that guy in my platoon ?”. This is obviously figurative as I have never been in a war and cannot begin to imagine the reality of it. The idea in that in a combat unit if a grenade is thrown in the middle of the unit, the soldier who sees it first will cover it with his body to protect the rest of the unit. On the same idea soldiers rotate at the head of a column where you are more likely to be shot or to trigger a trap. These are only two examples of what lies behind the question. In the more peaceful environment of an agile team it means that when someone discovers technical debt, that person (that pair) needs to address it on the spot before it becomes a problem for others in the team. It means that the riskier tasks are rotated but affect a single member of the team at a time, limiting the risk that others will be affected.

This is why, for me, the question “Would I go to war with that guy in my platoon ?” summarises the kind of ethics I have come to like and expect in my team mates. A team that shows this kind of ethics is probably “jelling”. A team that shows opposite behaviours is probably falling apart and will drag the project in its wake.

I almost forgot. The ethics are important in an agile context because the process will emanate from them, making the team truly self organised, if not self directed.

I recently had to prepare a four minutes talk to present my favourite design pattern. Proxy was my choice and it made me realise a few things about it.

First that in the pattern (as it was defined by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides) the proxy has the same responsibilities as the real subject. This is a much more restrictive definition than what I had come to think as proxies. Also in one form or another the proxy is used to control (mediate ?) access to the real subject. Nothing new here, this is the plain definition.

I have three typical uses for the Proxy. First the access to remote objects which is now so commonplace that people hardly notice it. Second error handling around resources that can become unavailable (connections for instance). Finally access to pooled or cached instances of objects. In that last case it means the proxy can be left as a permanent placeholder and negotiate the use of the actual resource when needed (see of the impact of caching on your design). This last use also realises for dynamic objects what the flyweight pattern intends for static ones.

After thinking about I found why this pattern was significant for me. First is that proxies enable some degree of separation of concerns (say error handling versus and actual function), making the proxy a lightweight aspectisation (see how Spring does aop). Second I like it because it provides a great example of the usefulness of loose coupling through interfaces (another one being unit testing). Finally I find that proxies give a more dynamic feel to an application. They enable easy reconfiguration, hot swapping of objects and controls. This is particularly useful in a language like Java.

I will end this short presentation of my “favourite design pattern” with a note on dynamic proxies in Java. They do not necessarily lead to implementations of the proxy pattern. They can just as well be used to implement other structural patterns (from facade to bridge and adapter).

And that made me realise that design patterns are first and foremost about the intent. The implementation of different patterns can be the same, making them difficult to identify in a code base. One more reason to use sensible naming in your code I guess.

I had a discussion with a colleague the other day about the necessity of regression testing in fast release cycles.

I argued that manual regression is not necessary then, aside from the fact that it is typically in direct contradiction with the idea of having fast release cycles. Manual regression simply takes too much time for a system of significant size to allow monthly releases.

I believe firmly in automated regression. As a matter of fact we have three suites of regression test: unit tests, developer written integration tests and QA written silk tests. I also believe that the purpose of regression tests is to give the development team some confidence that no major regressions were introduced since the previous release. The word “major” in the previous sentence is very important, as is the expression “some confidence”. In other words the level of effort that goes into the regression testing is highly dependent on the appetite for risk the team has.

There are three good arguments for short release cycles and therefore limited manual regression.

First manually regressing a system is inefficient and should be limited to parts that cannot be automated (due for instance to the necessity to co-ordinate with other systems). Not automating the regression tests that can be costs development time (because feedback comes late in the process as bugs) and QA time. It obviously limits the ability to release in short cycles (our manual regressions currently take around three weeks for the entire QA team, leaving a week to test an entire month of development).

Second if the release cycle is short enough, most of the areas touched during the development cycle will have been tested as part of testing the new features. This may not be very thorough as far as regression goes but should uncover most “major” regressions and as a result improve the existing regression test suites.

Third having fast release cycles, including partially or fully automated deployment processes limits the risk created by regressions by providing an easy way to address them quickly, as they are discovered in the production system.

Should I underline that none of this is acceptable for critical systems. That is why the thoroughness of regression is dependent on the risk appetite of the system’s owners. An obvious follow-up question is “How do I know how much risk I am taking ?”.

There is usually a time in a project when the team forgets why things are done. Still for a long time we go on with the mechanics until someone questions them openly. The chicken continues to run.

It was obvious for all of us that, in their existing format, our retrospectives were useless. People would come up with well and not well items easily enough but the laundry list would never be acted upon and most of the subjects never discussed in depth. It was also clear to some of us that retrospectives have the potential to be greatly useful. Not to mention that if the retrospectives stop you might as well abandon agile altogether as evolution of the development process becomes impossible.

It so happened that a week before we tried to tackle this issue, Chris Sims facilitated a monthly meeting of the BayAPLN using group wisdom techniques. I set out to have the same applied to our retrospectives.

So for a few months now our retrospectives have followed the following process:
five minutes silence (and it is something to have silence in a meeting room) for each participant to think about something that was good or bad,
go a few times around the room to collect the notes (along with a brief explanation), one per participant and per round, highest priority should be given first, no duplicates,
once we have enough items, distribute voting stickers and get people to stand up, go to the board and vote (another nice addition to an otherwise sit down meeting),
tally the votes,
talk about the first few elected subjects,
after each subject assign an action item to one of the participants to either correct the problem or make sure good things continue to happen.

There are three important additional rules that make the process work. First each participant can put only one vote on a subject. Second the overall discussion is timed box. Third participants can ask for a conversation to be stopped (by means of a physical token like a stop or pause sign) and moved to a dedicated meeting.

The result of this is very positive as we are finally acting to continuously enhance our process. It also has an energising effect on the kick/close meeting by breaking the sit down routine.

I have to admit first that my first idea about this article was going to me a rant about how TDD goes well beyond what design by contract gives as I felt Bertrand Meyer in computer magazine dated August 2008 missed that point. Then I thought better of it and checked for resources through google and discovered that I am late to the party with at least three very intersting articles on thus very subject: DbC and Testing, more on TDD vs Design by Contract and Test Driven Development and design by contract, friend or foe. On the other hand I did not find a tool that was satifsfying to link TDD and DbC (don’t you love acronyms ?). So here is my attempt at a story describing what I would like.

Let’s imagine the following scenario. I need to write the function f of a service S.

That function will take an id and a number as input, find an object responding to interface O from a cache responding to interface C.

In a TDD manner I start writing the test. Discover in the process that S needs to know about a C. Stubs a C for the purpose of returning an O to my service. There I specify that the contract of the find message of C is that the passed id is not null, it returns a non null O or throws a NotFound exception.

Now I would like that the calls to the C stub would verify that contract to ensure I am testing in accordance to the contract. This way if the contract changes in a way that is incompatible with my assumptions, my test will break and signal the problem.

I would further like the contracts to be verified automatically against the instances of implementations of C throughout my development cycle. And I would like to be able to selectively keep the contract enforcement in my deployed product.

The net result is not to get rid of the defensive programming but to move it into a well defined area as a specific concern that crosscuts my software. I would even have the choice to test whether the contracts actually specify what I intended. It would also remove the necessity to test the defensive programming code in the same artefact as the functional code.

Why all that ? Because TDD, beyond creating regression test artefacts also is a way to specify the component being created and that therefore it would be nice to be able to capture in separate, executable artefacts the implied specifications made on the related components.

Anyone knows about such a tool ? Anyone wants to write it ? Am I crazy ?

Here are excerpts of a document entitled “A little guide to a little etiquette” that provides guidelines on what to avoid doing in a work environment designed to be collaborative.

“Walk, don’t yell”, “Use your inside voice”. Basically the designers understood that open floors have no barriers against sound. They also recognise that the natural behaviour when you see someone that you need help from is to ask them directly from the distance, shouting to get their attention. And they found a solution: change your natural behaviour (and do some exercise at the same time !).

Regarding personal items, “One rule of thumb is that if you can see it from a public space or across the room (…) it needs to go”. Interesting as from my perspective anything that is on a desk can be seen from a public space on an open floor. Note that there is nothing here to suggest that this treatment is reserved to offensive material. Well, okay Dilbert may be offensive to some. Also there is a ban on plants and tape to make sure that in reality you cannot display anything personal in this neatly designed, beautiful place.

Other gems include:
“A reminder that all holiday lights should be viewed at (edited name of a city spot), not your desk. Also please remove any holiday items by January 2.”
“Walk Around. Please use the main walkways and go around the perimeter of workstations instead of cutting through work areas.”
“Please avoid ground-hogging - peeking over the workstations walls in order to talk to co-workers.”

Aside from the fact that it is the first time I get to read a written form of the furniture police manual what strikes me, and I guess it is good news for us software professionals, is that even for interior designs we have decided to replace hardware (walls and doors) by software (behaviour). It is really unfortunate that the designer failed to understand that one of the prime examples for open floors, trading floors, are places made specifically to enable people to yell and sign at each other from across the floor.

Let me re-read Peopleware.

Or How thorough QA can lead to lower quality ?

Indeed this is a paradox but it appeared very clearly to me after observing a change in the QA style in our team. Before this realisation I would have happily argued that thorough QA is the best thing that can happen to a project. The paradox unfolds in three parts.

Firstly it makes it harder (and more importantly longer) to finish stories. I understand that it also avoids accepting low quality work. However the QA perspective is slightly different (and this is good) from an actual user’s. So thorough QA, uncovers plenty of minor problems that do not significantly impact user experience or that would be minor production issues. This generates frivolous work that delays the development of more important stories, bug fixes or refactoring, thereby reducing the overall quality of the product.

Second, after a while the QA style modifies the developer’s attitude. When QA is thorough there is a sense of false confidence in the QA process that generates sub par unit and integration tests. This in turns leads to more bugs. Also, when QA is too picky before accepting a story, it removes one of the incentives to write well tested code because the difference in terms of the time it takes to close a story with good tests or not is marginally different. This effect in general is all about removing quality ownership from the development team and transfer it to the QA function.

Lastly the combined effect of the first two is that velocity falls, the project gets behind, people (well mostly managers) get nervous and start sending the message that quality is not important. The first signal is to encourage QA to log a bug but pass the story. The second signal is to de-prioritise existing bugs, effectively ignoring them (as we all know bugs underneath a given critically are never fixed and fall into the priority blackhole).

And there you have the QA paradox. Does this resonate with your own experience ?

In agile literature, stories are sometimes described as a “place holder for a conversation”. The thinking behind it is that the client and the development team (developer and QA when there is one) will sit together and flesh out the details when the story is ready to play. This turns out to not be the entire story. First the story has to go through the estimation process. In order to give an adequate estimate (not necessarily a precise one) some details need to be available already. The story is enriched by this first conversation. Second because during the development phase, the specifics of the story may change due to technical difficulties, new ideas or the discovery of new use cases. This simply means the story evolves and changes and gets more precise as it approaches completion.

Unfortunately, in the team I currently work on, this introduces friction. The big question is how detailed should the story be for QA ? And even if we agree on this, how are discussions and changes recorded in the story ? Even more importantly, who records those changes ?

Enters BDD story writing.

I recently shared with the team an article describing a recommended story format introduced by BDD. I hope this can be used to address the different changes in the story. From the short narrative and user scenarii written by the client to the use cases discovered during the development and QA phases, the BDD format enables to easily enrich the story “just in time”. It enables to keep track of the state of a changing specification. It leaves most of the non critical specifics out and it does not record the full discussion but simply its results. It is also a format that is easily accessible to technical and non technical people alike, both for reading and writing. I will be sure to report on our progress using this technique.

All this helped me realise a few things about Agile Storytelling. As the title implies it is akin to the communication technique called storytelling. The story facilitates the conversation between people with very different backgrounds. It sets the common vocabulary. It also builds a common vision of the story’s results and in the long term of the product. A key difference is that the story is not a one way message, it is written by multiple people putting their own point of view in a language that is understandable by all.

Did I mention that an agile project without well written stories and a clear definition of “done” is bound to fail ?

Don’t miss Jay Fields’ excellent article about story estimation on infoq.

One thing that I think is particularly important: the accuracy of the estimation is not important, consistency is. The planning is done by combining velocity and estimations. The velocity already contains estimation errors so as long as your error is consistent everything is fine.