In the recent years we’ve seen a lot of discussions and good things about cloud computing – sharing platforms (PaaS), services (SaaS) and software thus optimizing the usage of computer resources.
This sharing of resources is important for making the software sustainable, and helps the companies to focus on what their business is about rather than on their IT infrastructure.
Measurement programs are no different – they are often a strategic value for companies, but they are not really something the companies want to spend their R&D budget for (at least not directly). So, how do we make it happen?
Well, we could use the same approach as in SaaS and PaaS and define MaaS (Measurement-as-a-Service) where we can reuse the knowledge across organizations and minimize the cost for working with the software measurement initiatives.
We’ve tried this concept with one of our industrial partners – Ericsson – and it seems that it works very well. You can read more about it in this article.
And the picture below explains a bit how this works.
Dashboards and all kinds of radiators are very popular in industry now. They allow the companies to disseminate the metrics information and to find the right way of visualizing the metrics.
In a recent article written together with Ericsson and Volvo Cars we have explored how to find the right visualization and we developed a model for choosing the dashboard – http://gup.ub.gu.se/records/fulltext/220504/220504.pdf.
The method quantified a number of dimensions of a good dashboard and provides a simple set of sliders that can be used to select the right visualization. The companies in the study have found it to be a good input to the understanding of what the stakeholders want when they say “dashboard”.
In the next steps we’re currently working on defining a quality model of KPIs – Key Performance Indicators. The first version has shown that it allows the companies to reduce the number of indicators by as much as 90% by finding the ones which are not of good quality.
In our recent work we have explored the possibility of validating that a measurement program is robust. We have worked with seven companies within the software center to establish a method and evaluate it. The results are presented in a newly accepted paper “MeSRAM – A Method for Assessing Robustness of Measurement Programs in Large Software Development Organizations and Its Industrial Evaluation” to appear in Journal of Systems and Software.
In short the method is based on a collecting the evidence that a measurement program contains elements which are important for the program to be able to handle changes. For example whether a measurement program has a dedicated organization working with it and whether the entire company is able to utilize the results from the measurement program.
The method is similar to the stress-testing of banks, so popular in the last decade.
The next step in our research is finding out which metrics the companies should use to assure the long-term robustness of the measurement program. stay tuned!
When working with companies in different projects I often get a question which metrics should an Agile software development team use. The answer is of course – It depends what your team does… and then a set of questions from my side follow. These questions are designed to make me understand about the activities which the team does, the activities downstream of the process, the product, the process, etc.
I’ve recently looked into this article where the authors make a review of metrics used in agile teams. Although I’ve had high hopes for them, I got a bit disappointed – they were more or less the same metrics as any other team uses.
Review article: http://www.sciencedirect.com/science/article/pii/S095058491500035X
However, metrics like the release readiness (see: our previous article from Ericsson) were not found….
I guess I need to search on…
Staron, Miroslaw, Wilhelm Meding, and Klas Palm. “Release Readiness Indicator for Mature Agile and Lean Software Development Projects.” Agile Processes in Software Engineering and Extreme Programming. Springer Berlin Heidelberg, 2012. 93-107.
A colleague from my division has written an excellent piece of empirical work on how the transition towards continuous deployment can be done in an Australian company.
Sincerely recommend reading the article at http://www.sciencedirect.com/science/article/pii/S0950584914001694
Predicting defects has been on my mind for a while and I’ve been collecting evidence of good metrics which can improve accuracy of predictions.
In this article Madeyski and Jureczko have found one more metric – Number of Distinct Committers (NDC) which seems to improve prediction models. The link to the full article is here: Which Process Metrics Can Signicantly Improve Defect Prediction Models? An Empirical Study.
The empirical evaluation includes 27 open source projects and 6 industry projects. It’s great that there is an increased body of evidence combining both the open source and the industrial projects. Especially that the results seem to be consistent.
Darko Durisic has done an interesting work on the evolution of industrial-class meta-models. The work has been accepted as full paper at SEAA (Software Engineering for Advanced Applications) Euromicro Conference.
Title: Evolution of Long-Term Industrial Meta-Models – A Case Study
Abstract: Meta-models in software engineering are used to define properties of models. Therefore the evolution of the metamodels influences the evolution of the models and the software instantiated from them. The evolution of the meta-models is particularly problematic if the software has to instantiate two versions of the same meta-model – a situation common for longterm software development projects such as car development projects. In this paper, we present a case study of the evolution of the standardized meta-model used in the development of the automotive software systems – the AUTOSAR meta-model – at Volvo Car Corporation. The objective of this study is to assist the automotive software designers in planning long term development projects based on multiple AUTOSAR meta-model versions. We achieve this by visualizing the size and complexity increase between different versions of the AUTOSAR meta-model and by calculating the number of changes which need to be implemented in order to adopt a newer AUTOSAR meta-model version. The analysis is done for each major role in the Automotive development process affected by the changes.
Stay tuned for the full version of the paper and congrats to Darko!
In our recent research we’ve looked at a number of ways on how to support software development companies in working with reliability modelling.
I have come across this article on how to choose a model – a sys rev. They authors look at a number of criteria and evaluate which ones are the most used in choosing models. Nice and interesting reading.
Link to full text
Which metrics are used by Agile teams?
Link to full text
I was browsing for articles to my new manuscript and encountered this nice piece of work. This article makes an overview which code metrics are used and why by agile teams. The needs are:
- Iteration planning
- Iteration tracking
- Motivating and improving
- Identifying process problems
- Pre-release quality
- Post-release quality
- Changes in processes and tools
The article of course mentions the metrics used in each category.
Article highlight: Empirical evidence on the link between object-oriented measures and external quality attributes: a systematic literature review
Link to full text
This article presents an interesting systematic review where the authors set off to look for evidence of correlation between OO metrics and quality. What I like about this paper:
- nice overview of which metrics suites exist for OO programs
- nice overview which external quality metrics are used
- essentially only 99 studies exist which have the right scope and quality
- the number of studies seem to be growing – even in the past 2-3 years
- the 20-year old CK suite is still the most popular one
Recommended reading for those who want to see which metrics are the best predictors, when and why.