Alice: Which way should I go? Cat: That depends on where you are going. Alice: I don’t know. Cat: Then it doesn’t matter which way you go.” Lewis Caroll, Alice in Wonderland
Many companies like to think that they need metrics to improve, which is often not true – they only improve when they show effects. This post is about my observations what kind of metrics lead to effects and how to think about the effects. So, choosing the right measure is mostly about choosing the right goal.
Recently I’ve been helping organizations to design measurement programs from scratch. Every time I encounter an organization which tries to establish the program, they start somewhere in the middle: not the end, not the beginning, but in the middle.
In order to illustrate this in a good way, I’ve looked at the Swedish Innovation Agency Vinnova’s effect-logic measurement: https://www.vinnova.se/globalassets/utlysningar/2018-02242/omgangar/mall-effektlogik.pdf916281.pdf (in Swedish).
In short, that kind of set-up of measurements requires two levels of measurements, but let’s start from the entire chain. The chain is presented in figure below. The figure is my own intepretation of Vinnova’s framework. In particular I add the goals, which are extremely important in measurement.
First, we need to define which goals we want to address. Then, we plan which activities we need to conduct to achieve these goals. Then we define the results from the project – what we deliver to address the goals. The results can be measured and quantified. The results also lead to some effects, which is often something that we can do thanks to these results. Finally, these new events and activities can lead to quantifiable effects.
So, how does this translate to the field of software engineering and software measurement? Let’s consider an example: we want to increase the quality of source code integrated to the main branch. We plan a project where we study the code review and testing practices. We deliver new methods which can speed up the code reviews. Our first measure, which we can turn into an indicator, is the duration of the review. The effect of this, as we anticipate, is the fact that the number of features delivered to the main branch will be higher (our effect measurement, or PI). Finally, the long term effect is that we can get more customers as we have more features.
We can easily identify measures and indicators here. This is all thanks to the fact that we put our story in a specific way – starting from the goal and ending in the effects.
So, instead of asking what to measure, first look into the goals and expected effects. Once you have these, it will be easy to identify the measures and indicators.