I have to admit first that my first idea about this article was going to me a rant about how TDD goes well beyond what design by contract gives as I felt Bertrand Meyer in computer magazine dated August 2008 missed that point. Then I thought better of it and checked for resources through google and discovered that I am late to the party with at least three very intersting articles on thus very subject: DbC and Testing, more on TDD vs Design by Contract and Test Driven Development and design by contract, friend or foe. On the other hand I did not find a tool that was satifsfying to link TDD and DbC (don’t you love acronyms ?). So here is my attempt at a story describing what I would like.

Let’s imagine the following scenario. I need to write the function f of a service S.

That function will take an id and a number as input, find an object responding to interface O from a cache responding to interface C.

In a TDD manner I start writing the test. Discover in the process that S needs to know about a C. Stubs a C for the purpose of returning an O to my service. There I specify that the contract of the find message of C is that the passed id is not null, it returns a non null O or throws a NotFound exception.

Now I would like that the calls to the C stub would verify that contract to ensure I am testing in accordance to the contract. This way if the contract changes in a way that is incompatible with my assumptions, my test will break and signal the problem.

I would further like the contracts to be verified automatically against the instances of implementations of C throughout my development cycle. And I would like to be able to selectively keep the contract enforcement in my deployed product.

The net result is not to get rid of the defensive programming but to move it into a well defined area as a specific concern that crosscuts my software. I would even have the choice to test whether the contracts actually specify what I intended. It would also remove the necessity to test the defensive programming code in the same artefact as the functional code.

Why all that ? Because TDD, beyond creating regression test artefacts also is a way to specify the component being created and that therefore it would be nice to be able to capture in separate, executable artefacts the implied specifications made on the related components.

Anyone knows about such a tool ? Anyone wants to write it ? Am I crazy ?

Here are excerpts of a document entitled “A little guide to a little etiquette” that provides guidelines on what to avoid doing in a work environment designed to be collaborative.

“Walk, don’t yell”, “Use your inside voice”. Basically the designers understood that open floors have no barriers against sound. They also recognise that the natural behaviour when you see someone that you need help from is to ask them directly from the distance, shouting to get their attention. And they found a solution: change your natural behaviour (and do some exercise at the same time !).

Regarding personal items, “One rule of thumb is that if you can see it from a public space or across the room (…) it needs to go”. Interesting as from my perspective anything that is on a desk can be seen from a public space on an open floor. Note that there is nothing here to suggest that this treatment is reserved to offensive material. Well, okay Dilbert may be offensive to some. Also there is a ban on plants and tape to make sure that in reality you cannot display anything personal in this neatly designed, beautiful place.

Other gems include:
“A reminder that all holiday lights should be viewed at (edited name of a city spot), not your desk. Also please remove any holiday items by January 2.”
“Walk Around. Please use the main walkways and go around the perimeter of workstations instead of cutting through work areas.”
“Please avoid ground-hogging - peeking over the workstations walls in order to talk to co-workers.”

Aside from the fact that it is the first time I get to read a written form of the furniture police manual what strikes me, and I guess it is good news for us software professionals, is that even for interior designs we have decided to replace hardware (walls and doors) by software (behaviour). It is really unfortunate that the designer failed to understand that one of the prime examples for open floors, trading floors, are places made specifically to enable people to yell and sign at each other from across the floor.

Let me re-read Peopleware.

Or How thorough QA can lead to lower quality ?

Indeed this is a paradox but it appeared very clearly to me after observing a change in the QA style in our team. Before this realisation I would have happily argued that thorough QA is the best thing that can happen to a project. The paradox unfolds in three parts.

Firstly it makes it harder (and more importantly longer) to finish stories. I understand that it also avoids accepting low quality work. However the QA perspective is slightly different (and this is good) from an actual user’s. So thorough QA, uncovers plenty of minor problems that do not significantly impact user experience or that would be minor production issues. This generates frivolous work that delays the development of more important stories, bug fixes or refactoring, thereby reducing the overall quality of the product.

Second, after a while the QA style modifies the developer’s attitude. When QA is thorough there is a sense of false confidence in the QA process that generates sub par unit and integration tests. This in turns leads to more bugs. Also, when QA is too picky before accepting a story, it removes one of the incentives to write well tested code because the difference in terms of the time it takes to close a story with good tests or not is marginally different. This effect in general is all about removing quality ownership from the development team and transfer it to the QA function.

Lastly the combined effect of the first two is that velocity falls, the project gets behind, people (well mostly managers) get nervous and start sending the message that quality is not important. The first signal is to encourage QA to log a bug but pass the story. The second signal is to de-prioritise existing bugs, effectively ignoring them (as we all know bugs underneath a given critically are never fixed and fall into the priority blackhole).

And there you have the QA paradox. Does this resonate with your own experience ?

In agile literature, stories are sometimes described as a “place holder for a conversation”. The thinking behind it is that the client and the development team (developer and QA when there is one) will sit together and flesh out the details when the story is ready to play. This turns out to not be the entire story. First the story has to go through the estimation process. In order to give an adequate estimate (not necessarily a precise one) some details need to be available already. The story is enriched by this first conversation. Second because during the development phase, the specifics of the story may change due to technical difficulties, new ideas or the discovery of new use cases. This simply means the story evolves and changes and gets more precise as it approaches completion.

Unfortunately, in the team I currently work on, this introduces friction. The big question is how detailed should the story be for QA ? And even if we agree on this, how are discussions and changes recorded in the story ? Even more importantly, who records those changes ?

Enters BDD story writing.

I recently shared with the team an article describing a recommended story format introduced by BDD. I hope this can be used to address the different changes in the story. From the short narrative and user scenarii written by the client to the use cases discovered during the development and QA phases, the BDD format enables to easily enrich the story “just in time”. It enables to keep track of the state of a changing specification. It leaves most of the non critical specifics out and it does not record the full discussion but simply its results. It is also a format that is easily accessible to technical and non technical people alike, both for reading and writing. I will be sure to report on our progress using this technique.

All this helped me realise a few things about Agile Storytelling. As the title implies it is akin to the communication technique called storytelling. The story facilitates the conversation between people with very different backgrounds. It sets the common vocabulary. It also builds a common vision of the story’s results and in the long term of the product. A key difference is that the story is not a one way message, it is written by multiple people putting their own point of view in a language that is understandable by all.

Did I mention that an agile project without well written stories and a clear definition of “done” is bound to fail ?

Don’t miss Jay Fields’ excellent article about story estimation on infoq.

One thing that I think is particularly important: the accuracy of the estimation is not important, consistency is. The planning is done by combining velocity and estimations. The velocity already contains estimation errors so as long as your error is consistent everything is fine.

In an attempt to finance the renewal of my website service I set up a store on cafe press with my very own branded items ! Ok, I will admit it, it is really for the fun because I cannot really hope to gain a lot of money from this (and certainly a lot less than cafe press itself…).

Still if you find these products attractive, feel free to show your support.

Since I started exploring the world of agile methodologies I have come to understand that software engineering is all about communication. Even typing code is all about communicating your desire to the computer and communicate your intent to fellow developers who will come after you. Every time a communication break down happens, things will be painful. The lucky ones will be gone from the project before the pain becomes noticeable.

This in turn led me to appreciate the meaning of ‘people over process’ found in the agile manifesto. Sure advocates a number of important and useful practices. Most of them are aimed at improving communication at one level or the other (Stand-ups, pair programming, tests, continuous integration, stories, show and tells, retrospectives to name a few). None of that will be useful unless the people involved in the project actually make it work. This is not specific to Agile but to any process that is not mechanised.

So it takes a particular type of people (from stake-holders to developers) to make an agile project work, to get the process running and working. What I am starting to think is that regardless of the process initially chosen, these people would make it succeed anyway, unless they were strongly constrained by red tape and armed guards.

So is it the agile methodology that makes the project succeed or is it that the people who like agile environments are better at delivering ? In other words is the agile process immanent or transcendent ?

César touched on hiring and promoting a while back. This came in conjunction with a number of other things that I have heard or lived in the recent past. It is all about the “right” person.

What is the “right” person ? Is she the person that satisfies all your criteria ? Will your criteria become obsolete as the environment evolves ? Or even worse you will discover that you had the wrong criteria ? What if the job you envisioned for the person has changed before they could actually start ? And if the “right” person does not want to work with you, will you settle for a “wrong” one ?

Hiring is all about compromise. The firm needs to make a compromise between diversity of opinions and common values. It needs both to be innovative. Compromise because a job always needs a mix of qualities a fine balance between expertise, character and ethics. There is also an important compromise between immediate results and long term contributions by the new hire. For instance, do you prefer to hire a cheap graduate that you will have to train but you will also get to grow or a seasoned veteran that has done the same thing ten times and will do it again for you ?

Earlier in my career I tended to interview people as a reflection of myself rather than as separate contributors. It turns out no one like me wanted to work with me. As I have aged I now realise that a job definition will always be stereotypical. The person hired to fill it will largely influence its actual shape. And the social group (team, department, company) is the result of the co-evolution of all the persons that compose it. Therefore the “right” person would be right in the situation as it was before you hired her. There is no telling what the situation will be after.

So as a conclusion, if you are deterministic, the person you hire is necessary the right person. For the rest of us there is no such thing as the “right: person. In other words nobody is perfect and right implies a contextual perfection that does not exist. More importantly, looking for the “right” person clouds your judgement as a recruiter. You have to be agile as a recruiter as well.

César over at non linear enterprise is spot on about feedback loops.

That is what so-called reenforcing practices are about. Pairing introduces a very fast feedback loop on code writing. Automated regression tests provide a fast feedback loop on development. Retrospectives are a quick feedback loop on the project.

All this would tend to indicate that Agile is actually a more tightly controlled process than many other development practices. Even more interesting the control is exercised at the time and on the rythm that is most appropriate to the controlled process. Finally most of this control happens without external intervention (which will insecure many managers) which removes the controller single point of failure. It also makes it possible for the process to adapt and self repair.

But beware that feedback loops can also be a factor of instability…

I finally had the revelation, the answer to that puzzling fact that, no matter how much effort and faith a company puts into its transition to Agile methods, it fails. Well it is a bit blunt. Some actually succeed. And others manage to maintain a pretence of success for years.

I have been reading Peopleware recently and found the answer there. I am sure it exists in other places as well. It is not as if it was a hidden treasure of wisdom or a lost manuscript by aristotle. It is the well known fact that change is difficult. So difficult actually that the authors of Peopleware advise to effect only one change at a time ! And here is my realisation. To transition to Agile takes more than a single change. You have to change how people work together collectively, how they work individually, how information flows inside and out of the team and even what type of communication is needed. Any of these changes is challenging by itself. These categories may even hide multiple changes (switching to Test Driven Development and Pair Programming for instance).

Interestingly, this realisation ties perfectly with an open space discussion that happened at the BayAPLN a while back where we discussed the strategies to effect a successful transition to Agile. We came to the conclusion that it was easier (necessary ?) to foster change agents (positive deviants). Another successful approach was to sandbox agile practices. Agile sand boxing is essentially an attempt to transition small isolated (geographically or functionally) groups that are willing to become agile. For obvious reasons, groups that are not sand boxed are likely to be crushed by the rest of the company. The sand boxed group is freer to experiment with practices and to be allowed to progressively install new standards. The rate of possible change is also arguably improved.

So now I think I have linked the reason for failure and the path to success. Or am I missing something ?