I recently had a pressing need to return to one of my youth’s favourite pass time: the jigsaw puzzle. I, shamefully, never realised that this was related to the basics of problem solving. So here are my observations from the latest jigsaws I did.

You need a big picture to build a mental representation of the problem. This enables to make sense of the various pieces and enables to organise them in groups. The big picture also drives the resolution strategy such which elements to tackle first, the big groups and how they relate to each other.

You have a frame. That is a given constraint of the problem. It is usually useful to build the frame first but it is not necessary. The simple knowledge it exists is fundamental.

When I am assembling a puzzle I typically go trough iterative sorting, producing finer and finer sub groups with the pieces. This is enabled by the big picture and by the knowledge that is acquired over time of how the pieces relate to the big picture.

I proceed to the assembly in groups that are easily identified. I do not force the issue of finishing a group though. If a lot of pieces are missing, it is an opportunity to iterate on pieces sorting. If only a few pieces are missing, I will opportunistically gather them in a future sorting operation.

Sometimes I build small groups without knowing exactly how or where they fit in the big picture. Such groups are attached to other groups later on, after some more overall progress has been done.

Sometimes I misplace a piece. This usually happens in uniform parts of the puzzle, trying too hard to put pieces with little correlation from the neighbouring pieces. This leads to rework. It happened to me recently that I discovered the misplacement when trying to put the last piece. Resolution requires a careful inspection of how the other pieces fit together.

When a group is too difficult or proves elusive (such as a large area of a single colour), I alternate pointed and organised attempts at finishing it and pauses to resolve other parts. This usually means that they are the last parts finished. Usually at that point a spectator would be able to relate the puzzle and the big picture.

Although puzzles are significantly better structured and simpler than real life problem, I find that I apply a lot of the same strategies in my daily work. What do you think ?

The Bay APLN was kind enough to enable its members to beta test the Group Coherence workshop that will be presented at Agile 2009 by César Idrovo and Joanna Zweig. And it was really good. I would not go as far as life changing (as others have reported) but it certainly helped make sense of what lies beyond team dynamics.

I will avoid spoiling the experience of future attendees by describing the workshop. The process and the tools you will be building through the workshop will help you facilitate the apparition of group coherence. And it is far from being an exact science for now.

My personal highlights were:

people need to know their roles,

heroes and martyrs are an obstacle because they do not leave enough space for others to contribute,

the normal ‘good’ team dynamics must exist,

some sense of purpose and pressure must exist,

practice is needed to transforms practices into reflexes.

This eventually led me to form the metaphor that group coherence is a crystallisation process. The chemicals (people) are at some temperature (group dynamics) and under some level of pressure (deliverable, urgency, social pressure). When temperature and pressure align (and unfortunately that alignment depends on the chemicals) a crystal is formed. It could be a snow flake or a diamond.

Now some may prefer the cuisine metaphor but in the end, it is all chemistry anyway.

This is a rebound from the openspace session I hosted at the BayAPLN event on April 21 2009.

III started by noting that it is not your money. I initially interpreted to mean that therefore we do not have to care. Except if you have been entrusted to manage it and you have a fiduciary duty to make the most of it. Which lead us to the two main themes that informed the question which we essentially held to be the two sources of technical debt. One was the managerial impact. The other is the professional integrity of the team.

Lack of planing, disregard for appropriate buffering on delivery dates, inability to motivate the development team are all very heavy contributors to the increase of technical debt. The primary reason is that they tend to strip the development team members of their professional integrity, part of which is the pride people put in a work well done. In essence those managerial attitudes are akin to asking team members to break the windows of the building they are making. Or to cut on the cost of material while building in a seismic zone. To that extend technical debt resembles more a ponzi scheme than an actual financial debt.

We found hope in the professional integrity of the team. This professional integrity means, in my mind, that a developer should leave the code in a better shape than he found it. Pushing further in that direction it means that repaying the technical debt is a deliberate opportunistic activity. In other words technical debt should not be repaid for the sake of it, but only when it causes friction in the current work. Its repayment becomes continuous and integrated in the work stream. However, a lack of professional integrity (whatever the reason) will let entropy creep into the system and burden it with technical debt.

The conclusion, for me is three folds. First I changed my definition of technical debt to “things that could be done better” (now not, not things we could have done better). Second, you should not have to care about technical debt. If you have to reserve time to address, something is broken and should be fixed first. Third some debt is good and even inevitable, don’t try to eliminate it for the sake of it.

I was reading Neal Ford’s excellent articles on the benefits of TDD, part 1 and part 2.

One thing that struck me however is the attempt to limit TDD to test first and how this lead to a somewhat dishonest presentation of the test after approach. In the first of these articles the test after approach is stopped before the refinement phase that is made explicit in the rest of the articles and is well worth reading.

It is unfortunate because the real benefits, in terms of quality, from TDD come from the feedback loop that is created between code and test. It is the iterative refinement of the test and the code that provides the real benefits. It is a chicken and egg thing. Neither comes first, what comes first is an initial design idea based on the problem statement. In Neal Ford’s articles, one of those design ideas is to extract the set of factors whereas the problem does not require it (it requires the sum of factors and in many cases only a partial sum).

Testing before or testing after is an axiomatic decision that provides its own set of benefits. First it emphasises that the developer has to think about the design separately from the code. To a large extend this opens creativity as the design will be less constrained by the existing code. It is an explicit permission to re-factor as needed. The second (maybe the biggest) benefit of testing first is, in my opinion, that it is a clear signal that tests are not considered as a second class deliverable but as a scaffolding and design tool.

So to summarise, TDD is a design method in which tests are a design tool (regression testing is a perk). Test first is a process element that helps initiate and sustain TDD.

The BayAPLN was organising an open space event on the theme “Spring Cleaning…”

I was a bit disappointed that my proposed session “Is Technical Debt the Dark Matter of Software ?” did not get any interest. My second entry “Why should we care ?” was luckier as it attracted III and Paul Beusterien and lead to a very interesting conversation summarised in the proceedings along with the other sessions.

I also thank Chris Sims for picking my badge at the raffle. The ThoughtWorks Anthology: Essays on Software Technology and Innovation (Pragmatic Programmers) now joins my reading list.

I was troubled a few days ago when I heard a development manager talking about strategic initiatives and “tactical” solutions. I felt very strongly that the implied meaning was low cost, low quality solutions.

I am very grateful to Tim over at thedwick for writing that we should essentially reconsider that implied meaning. I would go even further, every single solution implemented by a development team is tactical in nature. Tactical does not so much refer to scope or quality as it does to the amount of control one has on the outcome and the time scale.

Strategic is when you have some level of influence on the goals and outcome but no real control. Tactical is that you may have some influence on the goals and full control on the outcome through your actions (your actions therefore have to adapt to local conditions and oppositions and overcome them). A strategy usually results in plenty of tactical decisions and actions that do not need to be fully defined in the plan.

The second aspect of the difference between strategic and tactical is the time scale. Strategic plans are liable to change as the environment changes until the strategic goals are achieved. The strategy will also change based on the tactical situations reported. Some goals may be modified or even abandoned if it appears they have become too expensive to achieve. On the tactical side, the goal is usually firmly defined and achievable in a relatively short time frame (a few hours to a few days) which leaves little room for changes.

One corollary of these two aspects is that when a tactical goal is achieved there is no reason to come back to it to achieve it again but better. To use a military analogy, let us imagine that the goal is to take a house in a city. The unit blows a couple of houses around to remove resistance and then storms the objective. Three weeks later, after the city officials complained, the unit is asked to rebuild the two houses and asked to put back the opponents that were in them and then take the same house again without destroying any other buildings in the city. That is exactly what is usually implied when a development manager talks about a tactical solution.

PS: It will be obvious that, in the analogy, the tactical goal has actually changed but let’s keep it simple.

Not so long ago I realised how many of the agile characterisations done by opposing Agile and Waterfall do not apply in my experience. In other words I have never encountered a waterfall development management process. What I have seen is plenty of ad-hoc processes derived from the vague knowledge that in a project you have requirements, developments and tests in that order. These activities apply under Agile methods as well.

In my experience (other experiences may vary…) the opposition is really between Agile (a well documented set of self reinforcing principles and practices) and the processes that managers have been empirically evolving for years after possibly learned from their mentors. It is much harder to make these people abandon their home grown methodology and do a leap of faith. It is much easier to help them evolve it by teaching things from the Agile toolbox, from engineering practices (TDD, CI) to useful metrics (Work in progress, inventory) to practices (Stand ups, retrospectives). Many of these tools have measurable positive returns and some are a small evolution from the existing ad-hoc practice.

The deadly poison here is to attach those tools to a specific methodology, unless the managerial hierarchy itself is pushing it (à la salesforce.com). A formal checklist of what is needed before a project can be called agile feels like a bad idea. We should instead thrive to put in place a well chosen set of the proven tools that Agile helped popularise. That will bring immediate benefits without a leap of faith. You may not be Agile in the end but you, your team and your customers will be happier.

It looks as if I would love to work there, although they probably would not hire me…

See their post with a link to the slides.

I will try to avoid making excuses from now on.

A very nice hands on roadmapping workshop at the BayAPLN yesterday. This was a very successful meeting where members got to practice roadmapping on our very own community.

Thanks again to the CoCo and Scott from enthiosys for making it happen.

In particular check out the templates they provide. I started practising today on the simplest template and we will see how that turns out.

I even got lucky and won a book at the raffle !

I recently had a discussion with my newly appointed manager over his idea that having to redeploy the system in order to change its behaviour is bad and that configuration files would be a superior solution. As it happens our system is Spring contained so it does not lack configurability and we take advantage of that to support diverse environments. However that is not what he was thinking of. He was thinking of having the actual functions of the system being configurable. It took me a while to find a way to express why it is a bad idea and usually the result of a lack of design.

So let say you want to implement a function that in the context of the system sends an email to the system’s support team. Natural parameters of that function would be the subject and the body as these are known only during the course of the system’s operations. The subject, in this case is a configuration element. The configuration element here is the recipients email address. Depending on the circumstances you may hard-code it to, for instance, “support@digitalbrikes.com” and let the administrators of the e-mailing system figure out this address should map to actual people. You may make it a configuration parameter passed to the system at start-up time, for instance, “sales-support@digitalbrikes.com” or “web-support@digitalbrikes.com”. This can clearly be useful. Another solution would be to configure the actual list of people directly, for instance “denis@digitalbrikes.com, bob@digitalbrikes.com”.

So what the configuration does is it increases the number of parameters of a function. Say we call our function f(x,y), adding one configuration parameter means that now we have a function g(x,y,z). Overall this is not too harmful, especially in an object oriented context where the object would carry the configuration parameter. The development effort and testing effort remain similar.

Now let’s take a more complex example where two configuration parameters are used to determine the parameters of the executed function. So suppose that we have our g(x,y,z) function that sends an email to z. Let’s now suppose that our function is that we want to send emails with a given prefix to management in addition to support. We now have three configuration parameters: support recipient, management recipient and the prefix that will determine who the email goes to.

Now the situation is that we have a function h(x,y,t,u,v) to send our email. Clearly this is more complex to code and test. Lucky for us our particular function can be expressed as the composition k?g i.e. h(x,y,t,u,v) = g(x,y,k(t,u,v)), which is somewhat simpler to code and verify than a single five parameter function.

Throughout this post, it is obvious the configurability may be legitimate in the context of the system. What the examples illustrate is that it significantly and quickly increases the complexity of a system and that therefore it is expensive and should be used only when necessary. In other words reducing configurability should be a design goal (simplicity).