A few months back, the team I work in was trying to introduce agile methodologies, such as test driven development and pairing. We were all very experienced professionals but none had practiced agile methodologies in a structured manner.

It so happens that some of us were fully convinced of their benefits (and informal practitioners). Others were skeptics. And one was firmly opposed to these practices because he had been very successful doing exactly the opposite and he felt hindered. It is only after he left the team that I realized how valuable this particular member was. Not only because of his expertise but for another very strange reason.

By constantly complaining and criticizing our practices he helped us refine them and remind everyone in the team what they were. The later proved invaluable as we were constantly falling back to our old habits and slowly decaying our advertised practices. His criticism was providing every member of the team with a strong sense of what we wanted our identity to be. The team’s social pressure was very useful to keep everyone in line. More so than any manifesto or charter. I believe this is akin to the kind of effect pairing has on code quality.

When he left, we started abandoning our best practices even faster. Positive reminders did not achieve the same effect. Granted, other changes came in the way of pairing in particular. It also happens that he actually started to see valuable things in our practices which made him a less steadfast contradictor.

So there you have it, maybe it takes a barbarian for greeks to be greeks. But beware that barbarians may in turn become greeks.

The financial world is abuzz with what happened at Société Générale. Many among the professionals are surprised by the size of the loss but not at all by its possibility. Why ?

In the past 10 years the systems security has greatly improved: requirements on password composition, frequent renewal of passwords have done a great deal for that. A bit deeper a lot of things are logged, traced, recorded: phone conversations, e-mails, application use. Even deeper, applications support things like two person checks, roles, permissions to ensure that users are allowed to do only what they are supposed to and get appropriate approvals for this. I would not dare to contend that these protections and safeguards are completely foolproof but for most of the applications I know, hacking them is beyond the reach of a master in finance.

So what gives ? I will give a few examples.

If in a back-office the official procedure is to use cross checks to avoid errors (pairing in agile terms), what really happens most of the time is that operators will exchange their passwords to enable them to do the cross check alone. Why would someone do that ? Because otherwise everybody would have to be here all the time, for very long days, taking no vacations, not being sick.

In a front-office, as noted by some commenters, passwords are on yellow sticky notes inside drawers or underneath keyboards. Why would someone do that when they signed a paper saying they should not ? Because in many applications, other traders cannot act on each other’s trades. Or the procedure to be able to do so is too complex. Or one of them started a trade and it finalizes while he is out getting a cup of coffee. His colleagues naturally cover for him and breach those security checks to accommodate team work.

I will not pile on with users that have permissions they should not, permissions that are not timely maintained when a user changes role, the heavy use of excel for critical things, failures in systems and missing checks.

To simplify, back-offices are the production line and front-offices the sales/engineering force. I have not yet spoken of the risk control part because I have little experience in this area. Believe it or not, risk control has considerably matured and is now very important in many firms. It always hold veto power over the front-office. Unfortunately, without the back-office risk control is half blind (as the SocGen example shows). The sub prime crisis on the other shows that risk models have a hard time keeping-up with financial innovation.

So what is the point in this ?

A lot of ink was devoted to the idea that information systems were tricked to produce unreasonable exposure. The reality is that it is the actual (as opposed to written) processes of the bank that were taken advantage of. The point is that when people cannot work under the rules and procedures they bend them or they give up. Either way problems of this nature will happen. Empowering workers to make appropriate changes to the way they work increases productivity and quality (such as better operations reports in this case). This is the realization at the core of Kaisen in the Toyota production system, this is also the core of Agile methodologies.

This is supposed to be a very flammable article so please light the fire. I attended two talks by Linda Rising one at a Bay Area Agile Leadership meeting, the other at QCon. That added to some other readings fueled the thoughts in that post.

I believe that all humans are naturally driven to perfection. It is the way the human brain works, using pattern and simplifications to accelerate reasoning. When people imagine something it is always perfect. At least initially. Or ideal. It is really unfortunate that many of us realized very early that real things are not perfect. Some may argue that this is the source of the religious fact but I am neither a theologian nor a philosopher.

Working with people, knowing that they are not driving for perfection proves beyond any doubt that they are not motivated and committed. I am not saying it is wrong, I think at some point everybody will stop believing and will try to find something different in which to project their drive.

Now, in many agile talks you hear people saying that one of the founding principles of agility is to abandon the idea that you can make a perfect product in favor of a reasonably good target. This however is only a half truth. It is true that you need to realize that the finished product will be the results of trades off, compromises and misunderstanding because that influences how you are doing things. On the other hand, Kaizen, continuous improvement of the process and the product are all signs of the drive for perfection. The proof ? No one gives the termination condition of these iterations. The termination will always be arbitrary. In other words, it is almost impossible to give a definition of good enough until you see it.

I would further contend that for each item of work, a motivated individual will always do as good as he can. Never good enough.

What are the implications of this for agility ? One is that I think it explains why there is not strong competition among agile team members. To compete you want to do things better than the other team members. If you are trying to do things perfectly you will ask all the help you can. Is this what keeps agile teams motivated or is it a result of the motivation ? I think it is a self-sustaining feedback loop (or something like that). Command and control typically does not achieve that because it is much harder to keep people motivated in that kind of environment which means that sooner rather than later workers will shift their perfectionist mind to other subjects.

I will finish this article with a saying ‘Better is the enemy of good’ which essentially means that you never achieve good if you are aiming for better. I would add that to achieve good you have to aim for perfect.

This was the title of last week’s meeting of the Bay Area Agile Leadership, featuring James Shore. And it was really about iteration planning.

The presenter used an exercise (shortened because of time constraints) where the attendance is divided into multiple groups: teams in charge of building an iteration plan (that include product managers/ subject matter experts) and individual that represent stakeholders that are not part of the team. the purpose is for each team to gather requirements, set the priorities and estimations and come up with an iteration plan. Everyone then votes on the best plan. In this instance there were two competing teams.

Surprisingly to me, the team that I felt pay better attention to my requirements was the one that proposed the least appealing plan. In our case the teams were planning a 20 minutes talk on Agile. The key was to discover that stakeholders preferred an in depth talk rather than a breadth one (or as Keith Berry put it, preferred an iPod to a Zune). Interestingly the actual contents of the planned presentation were fairly similar but offered in a very different way. The in-depth offer gathered more buy-in. Did it leave up to expectations ? Somewhat. At least enough that I would attend the second iteration.

The main take away for me from this meeting is that there are (at least) three obvious ways of failing in Agile planning. The first one is to fail to gather requirements properly. In an Agile setup this can be corrected easily as long as the supposed requirements are approximately correct. The second one is that the iteration plan you propose may not be appealing to the stakeholders and lead to loss of interest on their part. This is probably the most difficult one to correct (especially in early iterations), as you get only one chance to make a first impression. The third one is to fail to align the delivery and the expectations set by the plan, which is to me personally what happened that day. I believe a project can survive the last one if ut delivered something good (although below expectations) and it does not happen every iteration.

After Cesar’s comments on You shall not compromise quality! and the obvious confusion between quality and qualities, I created the following simple analogy. I believe it conveys the point efficiently.

If your house is flooded, two plastic bags and some tape will be enough to keep your feet dry. The quality requirement (qualities) is that they do not have holes. The quality of the bag is measured by the number and position of the holes. If you are lucky they are all high on the bag and your feet may stay dry.

If you go fishing in a swamp, you should use specially designed boots. The quality requirements (qualities) are that they have no holes, are able to resist various environmental aggressions and that they are comfortable. The quality of the boots will be a combination of the comfort, sturdiness and water tightness.

Compromising quality is that you don’t apply the best practices, use the wrong tools (ever tried to knit boots or plastic bags ?), promising more than can be achieved (like trying to make a plastic bag with half the necessary plastic or making boots when the soles are above budget).

Final point, good boots are not cheap. Plastic bags are relatively cheap and can be used for lots of things but they easily break and end up polluting the landscape (who just thought Excel out loud ?!?).

Last week-end Agile in Action asked whether we (as professionals) should you always do what customer wants (and don’t skip the comments at least one of those is great, will give you pre canned arguments in such a discussion and its not mine) ? It concluded that if a customer wants to compromise quality the author would walk away. I would agree with him, when it is possible because it is very much the sign of a barbarian culture on the clients part.

The way I understand it,  quality is a measure of how well the software product finally delivered implements the desired properties (or qualities) and in particular the desired functionalities. In that context, compromising quality means that you are prepared to deliver a product that may not be fit for its intended purpose. Which generally really means they you want to go to the market as fast as possible, let the customer test (discovering in the process what they really use and how) and then fix the problems as fast as possible. In short a barbarian approach. I believe that stating it this way is actually acceptable because it means the client understands the implications of the choice to be as fast as possible (and it may even be ok to execute on this in very specific circumstances).

Obviously, this is exactly the kind of situations that Agile methodologies address best. They enable to discover the functionalities the client really cares about as well as how they should work. They enable to make the other -ilities emerge though refactoring when they become desirable attributes of the product. They enable to bring features to market as soon as they work (as opposed to before they are tested). They are more satisfying for both the client and the implementor.

In conclusion, rather than compromising quality for the things you need, drop the things you don’t. Unless your client wants to consciously sabotage the product and specify the bugs, truly making them features.

I referred some time ago to an article by Robert L Glass published in IEEE’s Software magazine entitled "Greece vs. Rome Two Very Different Software Cultures". You can purchase this article but I find it a bit expensive for a two pages column. It is based on The Olduvai Imperative by Peter DeGrace and Leslie Hulet Stahl. Unfortunately this book seems out of print (although it can be found from specialty book shops online).

The author of the article describes three work cultures. The greek one where workers are "(…) individuals or self motivated members of teams" that were behaving like contractors and the roman one where the worker is "(…) sacrificing himself for the good of the organization, giving up his individuality, and closely identifying with his group." are taken from the book. He adds a third one, the barbarian.

He goes on to describe the compared values of each culture (I would have to reprint the whole article to give them here and I don’t want to infringe on the IEEE and Robert L Glass copyrights). Let me go straight to the conclusion as it applies to Agility: "Greeks would fit pretty well in the Agile Camp, Romans would be working mightily to improve their CMM level, and barbarians would say "huh?" if you mentioned either one.".

The article finishes with an excellent tale (taken from the book) telling what many of us have witnessed for a long time. The roman produces a great deal of external activity and produces a lot of artifacts showing more productivity. He beats the greek who took time to think of a simple solution that did not display enough effort. But Robert L Glass adds that in reality the barbarian wins by coding like crazy, introducing tons of bugs and then saving the day by fixing them. The barbarian, in our current software culture will be celebrated as a hero.

I know it is a very sad story. I blame Microsoft Windows a lot for that, having ingrained in the popular culture that bugs and crashes happen and are a normal part of software. But that is only my personal nag.

This is a follow up to Cesar’s short comment on Agile, leaders and the principles of democracy.

First I want to stress that democracy is not necessarily a representative democracy. Representative democracy appeared fairly late to make democracy work in ‘large’ countries (Montesquieu was contending that democracy could not survive in a large country that is not isolated, like France). Anyway, it is a fact that representative democracy does not typically lead to the emergence of the best leaders except in times of dire need because it is a game of seduction and compromission (this is frenglish, see a definition here). So nominating managers democratically  may not lead to more efficient or useful managers. One of the problems noted by Tocqueville in the early american democracy was that elected officials were submitted to the whims of the people. That impaired their ability to sustain a strategic vision. This lead to the progressive lengthening of public mandates.

Second, one very important difference between a country and a company is that, following Cesar’s view there are two potentially conflicting interest that need to exercise power. One is the employees. The other is the shareholder. There is a well documented history on the conflicts between these two parties and the subsequent emergence of unions. Ultimataly, the decisions lie with the shareholders in a liberal economy. One possibility to solve this contradiction is to create a cooperative where the shareholders are also the employees. It is a case when Cesar’s vision can be realized.

Which leads to my primary thought on the Agile company: it is the shareholders that impose it from the top. I believe it is in the shareholder’s interest to create a somewhat agile environment for his company. For it to happen, the shareholders must take responsibility to impose their rules and criteria on the management they are supposed to designate. This means the shareholders have to impose transparency and verify the integrity of the data and the decisions from the top. I believe that in such a company there would not need to have a lot of management layer. The board of directors could do most of the senior management work because in such a company their primary role would be to enable team collaboration at the level it needs to happen. Management would then be scaled back to what it essentially should be, an administrative support function stripped of most of the power that it attached to itself thanks to the command and control model.

I am fully aware that this is a bit of a day dream although I am pretty certain that some companies approach this model (the shallower the hierarchy, the closer). I don’t know if it can be generalized or even if it can be applied to all business models. And to be quite frank I am probably out of my depth here and I can feel a lot of sharp teeth nibbling on my feet so I will stop here.

A day passsed.

After writing those lines, I decided to document myself a little bit more and found two articles on self organizing work places: Volvo Uddevalla and GE Durham which are interesting to read in the context of this article and Cesar’s comment. However it got me confused because the Agile movement tends to claim to descend from the Lean production system (see Agile in Action for instance). These experiences are deemed opposed to Toyota’s approach but they are awfully close to what I would imagine an Agile company to be. Also to be noted that in both of these experiment, it required a special kind of employees to make it work.

This is by no means a summary of Eric Evans’ talk, only my main take away and some thoughts it triggered.

I have to admit that the term Domain Driven Design was a bit fuzzy and this talk did a good job at clarifying what it means. Essentially it means that when you build software you effectively build a domain model as well (whether implicit or explicit) and that you should pay attention to that.

In particular, the domain model you build is partial on the domain itself. It reflects some of the views you (and hopefully) your users, take on the domain. It serves the specific purpose of building a shared representation of the domain with your users. All that circled back neatly with agile and embedding the customer and all those good things.

Also I cannot pass over the row boat analogy through which Eric Evans helps explain why agility works more easily on small teams (he noted that boats have a maximum of eight rowers, in part because rowing requires perfect synchronization and that a single miss by one of the rowers throws everybody off rhythm). I would add a few notes to that. It is interested to note that boats of two and four are self organized whereas for eight someone needs to call the cadence. I would also like to reflect that it does not mean that boats of more than eight cannot exist (see the antiquity’s galleys). Just that to make them faster than eight will require a lot of training and that it may be difficult to assemble the individuals that fit together and that it may require more non rowing team members (who thought dead weight ?).

There was another greatly useful analogy with athletic relays which illustrated one pitfall of over engineering system interfaces but it would be too long to explain here.  The core of it is that you should engineer an interface that the other system will be able to handle reliably. If it is a csv file exchange then so be it.

Anyway, a final point before the presentation was interrupted by a fire alarm, was that there will always be multiple models of the same domain. That these models are more or less compatible and that you may need to translate between them. As it is, I have first hand experience in that with FpML, Calypso and others and I will add this, if you need to translate your model into another, make sure they are not too different or it will cost you a lot of time and effort which is exactly what you are trying to reduce by defining your domain specific model.

What is the link with strategic design in all that ? Well the premise that not all of a large system will be well designed. Therefore make your domain model explicit helps focus on the core of the system. If peripheral parts of the system are not as well designed, it is not so important.

Kent Beck’s opening keynote was about "trends in agile". This is in no way a summary of his talk but mostly my comments on things I found interesting or debatable.

He started by noting general business trends towards accountability, responsibility, transparency, relationships and how they are reflected in software development. I am somewhat surprised that he believes these are new trends. In the business to business relationship they were the cornerstone of the merchant empires (Netherlands, UK, Venice and others) and helped shape the western model of societies. It is true that there is a trend towards these in a more non business way (towards society as a whole). But, in particular, it has yet to become a real trend inside corporations (although, following Galbraith in particular,  responsible shareholders should push for it top to bottom)

The main point I got out of the keynote, which is extraordinarily important is the realization that software development is not longer the realm of anti-social wizards. That software developers are expected to behave like business partners. I believe in some places this has happened a long time ago (in many of the financial institutions I know in particular because technical wizardry is overshadowed by financial wizardry). But it feels like a valid point. I am not sure that it is a good thing either if it is  the result of ignorance (egg head bashing style) rather than of a better technological education.

Some of the questions also yielded useful reminders:

• Agile principles are more important than practices (I take it to mean that what makes you agile is the principles behind the practices you apply),
• When applying agile principles in distributed teams, it is crucial to ensure face to face and social time between the distant members once in a while (something that I have personally found to be very true as it gives you a much better understanding of the values and personalities of your colleagues).