I thought I would share my new preferred tool: sonar.

It is a web application that enables to visualize and exploit the results of static analysis tools, tests and dependency analysis tools. It gives a fantastic view of the overall quality of your code, highlights the problem areas and even enable you to visualize the highest priority items.

My favorite piece is the technical debt optional plugin. With some effort you can probably make it somewhat accurate for your particular circumstances. Which would be somewhat missing the point. Ideally you would want to use it as a benchmark value across organizations as the appetite for quality is already reflected in the rules you choose to include in the analysis.

And did I mention it is effortlessly populated from maven ?

I wrote a long time ago about how I would like these two tools to be combined.

In the end I gave it a shot and just posted that on github. I have not tried it in real life yet but I will soon. The README explains the intent and the design assumptions. Check the tests for some examples.

On the other end, nothing is more than valuable than your input so please let me know what you think about it.

Over the past few months I have grown to like the ‘given a’ syntax more and more (described here). So when I recently reactivated my Ruby On rails home project (see brikeyard deployed on heroku), I thought I would add that syntax to my rspec examples rather than use before(:each).

I have not spent too much time thinking about where to put this code. Nor did I spend much time peeling away the nested modules inside rspec. Ideas to make this better are very welcomed. I would also love to collect some feedback on this before I make a fool of myself by proposing its inclusion in rspec proper. Thanks in advance for your help !

Here is how the setup looks like in one of the specs:

describe :something do
    given {
      a StatisticalControl
      a WorkItem, {:lead_time => 1}
      another WorkItem, {:lead_time => 3}
      a ProcessStage, {:work_in_process => [a_work_item, another_work_item]}
      a ControlChart
      @project = Project.new
      @project.stub!(:lead_time_control).and_return(a_statistical_control)
      @project.stub!(:closed_stage).and_return(a_process_stage)
    }

    it "should do something useful" do
		(...)
    end
end

And here is what I added to my spec_helper.rb file:

def a(type, stubs ={})
  stub = mock_model(type, stubs)
  member_name = type.name.gsub(/[A-Z]/) {|match|
    "_#{match.downcase}"
  }
  instance_eval("
        @a#{member_name} = stub
          def a#{member_name}
            return @a#{member_name}
          end
      ")
end

def another(type, stubs ={})
  stub = mock_model(type, stubs)
  member_name = type.name.gsub(/[A-Z]/) {|match|
    "_#{match.downcase}"
  }
  instance_eval("
        @another#{member_name} = stub
          def another#{member_name}
            return @another#{member_name}
          end
      ")
end

class Spec::Rails::Example::FunctionalExampleGroup
  class << self
    def given &block
      before(:each, &block)
    end
  end
end

class Spec::Rails::Example::ModelExampleGroup
  class << self
    def given &block
      before(:each, &block)
    end
  end
end

Please let me know what you think. I welcome suggestions to improve.

In our process we have started using two metrics in addition to the cumulative flow diagram indicator. These metrics are tracked against target values to measure if the process is in statistical control.

The first metric is the lead time. It measures the time in days between the moment a work item enters our process and the moment it is declared finished and exists the process.

The second metric is the throughput period (the inverse of the throughput rate) which is the time in days between the end dates of successive work items. If we note two cards C1 and C2 with start dates SD1 and SD2 and end dates ED1 and ED2, then the lead time for C1 is (ED1 – SD1) in days and the cadence for that period is ED2 – ED1.

Over a period of time these measures can be plotted against a target mean (10 days of lead time and 4 days of throughput period) and a target deviation (5 days for the lead time and 2 days for the throughput period). These targets can be changed independently as our process matures. Improvement to the means benefit the delivery speed. Improvements to the deviation benefits the predictability.

Finally you will find here the model I use (for Apple’s Numbers) to visualize these process metrics.

I recently had a pressing need to return to one of my youth’s favourite pass time: the jigsaw puzzle. I, shamefully, never realised that this was related to the basics of problem solving. So here are my observations from the latest jigsaws I did.

You need a big picture to build a mental representation of the problem. This enables to make sense of the various pieces and enables to organise them in groups. The big picture also drives the resolution strategy such which elements to tackle first, the big groups and how they relate to each other.

You have a frame. That is a given constraint of the problem. It is usually useful to build the frame first but it is not necessary. The simple knowledge it exists is fundamental.

When I am assembling a puzzle I typically go trough iterative sorting, producing finer and finer sub groups with the pieces. This is enabled by the big picture and by the knowledge that is acquired over time of how the pieces relate to the big picture.

I proceed to the assembly in groups that are easily identified. I do not force the issue of finishing a group though. If a lot of pieces are missing, it is an opportunity to iterate on pieces sorting. If only a few pieces are missing, I will opportunistically gather them in a future sorting operation.

Sometimes I build small groups without knowing exactly how or where they fit in the big picture. Such groups are attached to other groups later on, after some more overall progress has been done.

Sometimes I misplace a piece. This usually happens in uniform parts of the puzzle, trying too hard to put pieces with little correlation from the neighbouring pieces. This leads to rework. It happened to me recently that I discovered the misplacement when trying to put the last piece. Resolution requires a careful inspection of how the other pieces fit together.

When a group is too difficult or proves elusive (such as a large area of a single colour), I alternate pointed and organised attempts at finishing it and pauses to resolve other parts. This usually means that they are the last parts finished. Usually at that point a spectator would be able to relate the puzzle and the big picture.

Although puzzles are significantly better structured and simpler than real life problem, I find that I apply a lot of the same strategies in my daily work. What do you think ?

This is a rebound from the openspace session I hosted at the BayAPLN event on April 21 2009.

III started by noting that it is not your money. I initially interpreted to mean that therefore we do not have to care. Except if you have been entrusted to manage it and you have a fiduciary duty to make the most of it. Which lead us to the two main themes that informed the question which we essentially held to be the two sources of technical debt. One was the managerial impact. The other is the professional integrity of the team.

Lack of planing, disregard for appropriate buffering on delivery dates, inability to motivate the development team are all very heavy contributors to the increase of technical debt. The primary reason is that they tend to strip the development team members of their professional integrity, part of which is the pride people put in a work well done. In essence those managerial attitudes are akin to asking team members to break the windows of the building they are making. Or to cut on the cost of material while building in a seismic zone. To that extend technical debt resembles more a ponzi scheme than an actual financial debt.

We found hope in the professional integrity of the team. This professional integrity means, in my mind, that a developer should leave the code in a better shape than he found it. Pushing further in that direction it means that repaying the technical debt is a deliberate opportunistic activity. In other words technical debt should not be repaid for the sake of it, but only when it causes friction in the current work. Its repayment becomes continuous and integrated in the work stream. However, a lack of professional integrity (whatever the reason) will let entropy creep into the system and burden it with technical debt.

The conclusion, for me is three folds. First I changed my definition of technical debt to “things that could be done better” (now not, not things we could have done better). Second, you should not have to care about technical debt. If you have to reserve time to address, something is broken and should be fixed first. Third some debt is good and even inevitable, don’t try to eliminate it for the sake of it.

I was reading Neal Ford’s excellent articles on the benefits of TDD, part 1 and part 2.

One thing that struck me however is the attempt to limit TDD to test first and how this lead to a somewhat dishonest presentation of the test after approach. In the first of these articles the test after approach is stopped before the refinement phase that is made explicit in the rest of the articles and is well worth reading.

It is unfortunate because the real benefits, in terms of quality, from TDD come from the feedback loop that is created between code and test. It is the iterative refinement of the test and the code that provides the real benefits. It is a chicken and egg thing. Neither comes first, what comes first is an initial design idea based on the problem statement. In Neal Ford’s articles, one of those design ideas is to extract the set of factors whereas the problem does not require it (it requires the sum of factors and in many cases only a partial sum).

Testing before or testing after is an axiomatic decision that provides its own set of benefits. First it emphasises that the developer has to think about the design separately from the code. To a large extend this opens creativity as the design will be less constrained by the existing code. It is an explicit permission to re-factor as needed. The second (maybe the biggest) benefit of testing first is, in my opinion, that it is a clear signal that tests are not considered as a second class deliverable but as a scaffolding and design tool.

So to summarise, TDD is a design method in which tests are a design tool (regression testing is a perk). Test first is a process element that helps initiate and sustain TDD.

It is a given that agile practices require an open and trustful environment to succeed. It is also given that agile practices help promote such an environment. But in a development project it is hardly enough.

In a development team, I like to characterise the level of trust I need to be comfortable with the simple question “Would I go to war with that guy in my platoon ?”. This is obviously figurative as I have never been in a war and cannot begin to imagine the reality of it. The idea in that in a combat unit if a grenade is thrown in the middle of the unit, the soldier who sees it first will cover it with his body to protect the rest of the unit. On the same idea soldiers rotate at the head of a column where you are more likely to be shot or to trigger a trap. These are only two examples of what lies behind the question. In the more peaceful environment of an agile team it means that when someone discovers technical debt, that person (that pair) needs to address it on the spot before it becomes a problem for others in the team. It means that the riskier tasks are rotated but affect a single member of the team at a time, limiting the risk that others will be affected.

This is why, for me, the question “Would I go to war with that guy in my platoon ?” summarises the kind of ethics I have come to like and expect in my team mates. A team that shows this kind of ethics is probably “jelling”. A team that shows opposite behaviours is probably falling apart and will drag the project in its wake.

I almost forgot. The ethics are important in an agile context because the process will emanate from them, making the team truly self organised, if not self directed.

I have to admit first that my first idea about this article was going to me a rant about how TDD goes well beyond what design by contract gives as I felt Bertrand Meyer in computer magazine dated August 2008 missed that point. Then I thought better of it and checked for resources through google and discovered that I am late to the party with at least three very intersting articles on thus very subject: DbC and Testing, more on TDD vs Design by Contract and Test Driven Development and design by contract, friend or foe. On the other hand I did not find a tool that was satifsfying to link TDD and DbC (don’t you love acronyms ?). So here is my attempt at a story describing what I would like.

Let’s imagine the following scenario. I need to write the function f of a service S.

That function will take an id and a number as input, find an object responding to interface O from a cache responding to interface C.

In a TDD manner I start writing the test. Discover in the process that S needs to know about a C. Stubs a C for the purpose of returning an O to my service. There I specify that the contract of the find message of C is that the passed id is not null, it returns a non null O or throws a NotFound exception.

Now I would like that the calls to the C stub would verify that contract to ensure I am testing in accordance to the contract. This way if the contract changes in a way that is incompatible with my assumptions, my test will break and signal the problem.

I would further like the contracts to be verified automatically against the instances of implementations of C throughout my development cycle. And I would like to be able to selectively keep the contract enforcement in my deployed product.

The net result is not to get rid of the defensive programming but to move it into a well defined area as a specific concern that crosscuts my software. I would even have the choice to test whether the contracts actually specify what I intended. It would also remove the necessity to test the defensive programming code in the same artefact as the functional code.

Why all that ? Because TDD, beyond creating regression test artefacts also is a way to specify the component being created and that therefore it would be nice to be able to capture in separate, executable artefacts the implied specifications made on the related components.

Anyone knows about such a tool ? Anyone wants to write it ? Am I crazy ?

Or How thorough QA can lead to lower quality ?

Indeed this is a paradox but it appeared very clearly to me after observing a change in the QA style in our team. Before this realisation I would have happily argued that thorough QA is the best thing that can happen to a project. The paradox unfolds in three parts.

Firstly it makes it harder (and more importantly longer) to finish stories. I understand that it also avoids accepting low quality work. However the QA perspective is slightly different (and this is good) from an actual user’s. So thorough QA, uncovers plenty of minor problems that do not significantly impact user experience or that would be minor production issues. This generates frivolous work that delays the development of more important stories, bug fixes or refactoring, thereby reducing the overall quality of the product.

Second, after a while the QA style modifies the developer’s attitude. When QA is thorough there is a sense of false confidence in the QA process that generates sub par unit and integration tests. This in turns leads to more bugs. Also, when QA is too picky before accepting a story, it removes one of the incentives to write well tested code because the difference in terms of the time it takes to close a story with good tests or not is marginally different. This effect in general is all about removing quality ownership from the development team and transfer it to the QA function.

Lastly the combined effect of the first two is that velocity falls, the project gets behind, people (well mostly managers) get nervous and start sending the message that quality is not important. The first signal is to encourage QA to log a bug but pass the story. The second signal is to de-prioritise existing bugs, effectively ignoring them (as we all know bugs underneath a given critically are never fixed and fall into the priority blackhole).

And there you have the QA paradox. Does this resonate with your own experience ?