article review – Page 6 – SE metrics (Software Engineering)

Data analytics in SE

https://www.sciencedirect.com/science/article/abs/pii/S0950584920301981

A few years ago, data analytics and big data were super popular in software engineering. In fact, they were a bit too popular, as many authors quoted big data because they had a diagram in the paper.

Fast forward to today and the situation is a bit different. We are more mature in using data in software development. We know that Big data is about the 5 Vs and that we can reason about it. We also know what providing the diagrams is not the same as using them to direct software development.

I found this paper when looking for literature for our new work on communication in software metrics teams. My colleagues study the communication and found that there can be several sources of confusion. Now, this paper is NOT about the confusion, but about prevalence of data analytics in software engineering. The working definition of Big Data Analytics is as follows in the paper: “Big data analytics is the process of using analysis algorithms running on powerful supporting platforms to uncover potentials concealed in big data, such as hidden patterns or unknown correlations”.

The paper poses three main research questions about the studies conducted in Big Data Analytics, about the approaches used and when they are used. I’m mostly interested in the second – which approaches are used. There, the authors pose three sub-questions:

RQ2.1: What types of analytics have been used in the ASD domain?

RQ2.2: What sources of data have been used?

RQ2.3: What methods, models, or techniques have been utilized in the studies?

In particular, the second one is the most interesting one – sources of data. There, the authors found that there are plenty. The entire table (Table 7 in the paper) is actually too large to quote, but let me just quote one of the categories: Source code and data model:

Source code
Ruby programs & Ruby on Rails
Java programs
Function calls
Code metrics
Development repository
Test case
Code quality
Application data schema

I recommend this as a good reading into the current state-of-the-art in data analytics in software engineering. I think we’ve matured a lot in the last decade as a community and that brings a lot of benefit. Our software development gets better and thus our software gets better.

From the abstract: In total, 88 primary studies were selected and analyzed. Our results show that BDA is employed throughout the whole ASD lifecycle. The results reveal that data-driven software development is focused on the following areas: code repository analytics, defects/bug fixing, testing, project management analytics, and application usage analytics.

Is confusion a factor when reviewing a code?

https://www.win.tue.nl/~aserebre/EMSE2020Felipe.pdf

Reviewing the code is an art. After working with the topic for a few years, we’ve realized that this is like reading a chat – one person responds to a piece of message sent by another person. The message often being the code and the response being the review comment. What we’ve discovered is that the context of the review is important as well as the possibility to ask questions. We even discuss having a taxonomy of these review comments to ease understanding of “where” in the review process one is at the moment.

This article caught my attention because it is about understading when a reviewer is actually confused when reading the code and making the comment. It’s a very nice piece of work as it combines code review comments analysis and surveys.

The results of the survey are interesting as they point out that the authors are confused much less than the reviewers – which is often caused by the fact that the comment is a response, while the code is the message. Quoting the paper: RQ1 Summary – Reasons for confusion: We found a total of 30 reasons for confusion. The most prevalent are missing rationale, discussion of the solution: non-functional, and lack of familiarity with existing code. We observe that tools (code review, issue tracker, and version control) and communication issues, such as disagreement or ambiguity in communicative intentions, may also cause confusion during code reviews.

Finally, I like the fact that the authors do a full systematic review on the topic and triangulate the results. This work will become a number one reading for my students in the programming course, which will teach them how important good code is!

From the abstract:

Results: From the first study, we build a framework with 30 reasons for confusion, 14 impacts, and 13 coping strategies. The results of the systematic mapping study shows 38 articles addressing the most frequent reasons for confusion. From those articles, we found 19 different solutions for confusion proposed in the literature, and nine impacts were established related to the most frequent reasons for confusion.

What’s going on in Deep Learning – a literature review (paper review)

https://arxiv.org/pdf/2009.06520.pdf

Image by Free-Photos from Pixabay

Deep learning in software engineering has been used extensively and there is a significant body of research about this topic. In this post, I would like to share my review of the recent systematic review on the use of DL in SE.

The interesting finding is the list of data sources for data for DL. Here, the source code data is prevalent. This is not surprising as we have GitHub with millions of repositories. The second largest is the repository metadata, again, for the same reason.

Although it is not surprising, it is really good to see this. I see it as a change in the research focus in the last 10 years. It shifted from the research on bugs and bug reports to the research on source code. I’m happy because helping out with the source code is the real improvement of the product, not an improvement of the process.

Another interesting finding is the use of natural language techniques as the most common ones, here I cite the paper: “Our analysis found that, while a number of different data pre-processing techniques have been utilized, tokenization and neural embeddings are by far the two most prevalent. We also found that data-preprocessing is tightly coupled to the DL model utilized, and that the SE task and publication venue were often strongly associated with specific types of pre-processing techniques.“

I recommend to read the article to get more insight into DL models used, which are quite many – from the standard cNN to more advanced GANs and AutoEncoders. Really nice!

Finally, the paper ends with a recommendation on how to use DL in other contexts, kind of flowchart. I do not want to copy it here, so I recommend to take a look at it in the paper: https://arxiv.org/pdf/2009.06520.pdf

Classifying SE research – an article review

https://link.springer.com/article/10.1007/s10664-020-09858-z?utm_source=toc&utm_medium=email&utm_campaign=toc_10664_25_5&utm_content=etoc_springer_20200904

Initially, I did not really think that I would put this article on the blog. I actually thought about using it in my writing advice page. However, I’ve read it and then I realized that it’s actually more suitable for this blog.

This article shows how we can classify software engineering research. It has a nice framework described in Figure 1. It organizes the framework around the concepts of who the main beneficiary is – e.g. human, system or researcher (yes, it is a different category!), type of research contribution and which research strategies are used.

It’s an article that complements the work of our colleagues from Lund University on the design of design science research studies and the construction of graphical abstracts.

Although the work seems to be obvious when you are a seasoned researcher, I need to be reminded sometimes about what kind of study that I want/need to conduct. Therefore I recommend this as a reading to both PhD candidates, master students and also advanced researchers. Using the classification scheme will definitely help us to understand each other better and to reduce the burden of paper reviews!

Action research in open source – a nice article

Image by Amílcar Vanden-Bouch from Pixabay

https://link.springer.com/article/10.1007/s10664-020-09849-0?utm_source=toc&utm_medium=email&utm_campaign=toc_10664_25_5&utm_content=etoc_springer_20200904

I’ve been looking out for good examples of articles about action research in software engineering for a while. There is a lot of those coming from the participatory design community and ethnography in software engineering.

This paper is an example of how one can conduct action research together with an open source community. It shows how to conduct the research while being part of the community and adds a new angle on the topic – how do we democratize the research design. In contrast to company-based development, an open source community is free to accept the new ways of working or not. Therefore, it can be challenging to make the action happen.

Figure 1 from the paper shows the process in more detail and I strongly recommend to take a look at it. It starts from the design of intervention, where community requirements, similar communities, best practices and problems are inputted. This similar communities precedence is new and important as it helps to leverage already adopted good practices.

The evaluation of the methodology was already done and it shows that it’s a valid and interesting new research method!

Abstract: Participatory Action Research (PAR) is an established method to implement change in organizations. However, it cannot be applied in the open source (FOSS) communities, without adaptation to their particularities, especially to the specific control mechanisms developed in FOSS. FOSS communities are self-managed, and rely on consensus to reach decisions. This study proposes a PAR framework specifically tailored to FOSS communities. We successfully applied the framework to implement a set of quality assurance interventions in the Robot Operating System community. The framework we proposed is composed of three components, interventions design, democratization, and execution. We believe that this process will work for other FOSS communities too. We have learned that changing a particular aspect of a FOSS community is arduous. To achieve success the change must rally the community around it for support and attract motivated volunteers to implement the interventions.

Technical debt from the perspective of practitioners – article review

https://link.springer.com/article/10.1007/s10664-020-09832-9?utm_source=toc&utm_medium=email&utm_campaign=toc_10664_25_5&utm_content=etoc_springer_20200904

Technical debt is a great metaphor in software engineering. It provides software engineers with the toolkit to communicate how bad design can affect the product in a long run, and how much it can cost to fix these problems. The metaphor has been implemented in many static analysis tools like SonarQube.

Despite its power in communicating, its not clear whether this metaphor is actually useful. It has some dark sides, which makes it a bit tricky to use it. For example, the “conversion” from a problem to the debt, e.g. lack of getter and setter methods to 0.5 days in debt, is one of these challenges. Its also not always clear which of the technical debt categories apply to which products.

What I like about this paper is that it presents a survey of technical debt. For example, it identifies the top causes of technical debt, such as:

deadlines,
inappropriate planning,
lack of knowledge, and
lack of well-defined process.

These challenges are present in most companies today, and the first two – deadlines and inappropriate planning – are often associated with start-ups and agile organizations. I recommend to take a closer look at the mindmap in the paper (Fig. 5) to dive deeper into the causes.

Quote from the abstract: We identified a total of 78 causes and 66 effects, which confirm and also extend the current knowledge on causes and effects of TD. Then, we organized the identified set of causes and effects in probabilistic cause-effect diagrams. The proposed diagrams highlight the causes that can most contribute to the occurrence of TD as well as the most common effects that occur as a result of debt.

Recommending refactoring via commit message analysis

Image by annca from Pixabay

https://doi.org/10.1016/j.infsof.2020.106332

In the process of reviewing code, we can identify refactoring pretty easy. We read the code, try to understand it and provide comments. In the understanding phase we also get ideas about possible alternative – why is this done this way?

Now, when writing the comments, we rarely have the time to refactor the code. In CI, this process of reviewing comes when we commit the code to the main branch and therefore we expect this to be delivered and used soon. So, it’s too late to refactor, we need to do it in the next iteration.

But the next iteration is the same, we need to deliver new functions, not “golden plate” the existing code, deliver it to the main branch, etc. When is the time for refactoring then? How do we document the possibilities and use them when we have a bit of time?

In this work, the authors look at the commit messages and identify refactoring possibilities for that, complementing the static and dynamic analysis of code. The method presented in the paper is based on the analysis of code from open source projects, the refactoring applied to the code and the analysis of the QMOOD quality attributes that were related to these commits.

The following quote from the paper explains a bit how the gist of the extraction of the refactored code works:

Identifying refactoring rationale has two parts. The first part is the detection of the files that are refactored by developers in a commit. The second part is the identification of changes in the QMOOD quality attributes then comparing these changes with the information in the commit message.

For the first part, we used the GitHub API to identify the changed files in each commit. In the second part, we compared the QMOOD quality attribute values before and after the commit to capture the actual quality changes for each file. Once the changed files and quality attributes were identified, we checked if the developers intended to actually improve these files and quality attributes. In fact, we preprocessed the commit messages and we used the names of code elements in the changed files and the changed quality metrics as keywords to match with words in the commit message. Once the refactoring rationale is automatically detected using this procedure, we continue with the next step to find better refactoring recommendations that can fully meet the developer’s intentions and expectations. In case that no quality changes were identified at all then a warning will be generated to developers that the manually applied refactorings are not addressing the quality issues described in his commit message.

Rule makers, Rule breakers – a book about societies

As part of my weekend reading I took on a book about cultures. Not a typical reading of mine, but I though I would give it a try. This book is about differences in cultures from the perspective of rules, laws, principles. It is a common knowledge that some societies are very relaxed (e.g. Scandinavia) whereas some are very strict (e.g. Singapore). Commonly, we also know that the loose societies are more creative, whereas the more disciplined ones are very, well, disciplined.

This book also makes a point that it’s not as simple as that. This is not a linear relationship and there is some golden middle. Just being a loose society does not guarantee innovativeness and just being strict does not guarantee discipline. People need a “lagom” (here is a good Swedish word, meaning just right, in the middle, just enough, not too much and not too little) set of rules and looseness in the society.

When I read the book I thought about well-functioning organisations. In the companies and teams that I visit (or used to, before the pandemic), I often saw teams that were working together well with some degree of looseness, but not completely without the rules. They tend to perform well and function well when all team members understand the goals and rules of the game. I’ve also seen teams that are not able to function at all. They do not respect each other, have no respect for rules and provide no support for each other. They are “too loose” and therefore they are groups, but not really teams. At the same time I saw teams where the narcissistic boss controls everything and everything needs to go through the boss. They do not produce much, trust me.

However, there is also one more aspect – it’s where you come from. I, for that matter, cannot work in a self-organising team. I just don’t know how to find my place. One of my friends told me once – “Either you lead, or you follow or you get the hell out of the way”. Well, I’m more for that kind of the rule. Following is nice and I like it, but self-organising is so-so.

I’m actually part of a self-organised team in one of my assignments. I don’t know what to do there, I rely on a friend from the team to tell me when is my turn to speak and if I should say something or not. He also tells me when it’s a good time to do things and when it’s just a discussion. I’m not providing the name of the friend, but I’m super happy that I have him!

To sum up, I really recommend to read this book as it provides a bit wider perspective on rules in societies than the most common books from the organisational theories. It is about a normal person like me trying to find a place in life (well, by now I would actually think that this is easier, but maybe I’m just wiser to realise certain things).

Thanks for listening and tun in for the next blog post.

Who should fix this bug – again?

Link to article: https://doi.org/10.1002/spe.2830

Assigning defects is a task that is not so much fun. Companies need to do that, but the persons who do it often change as the task is quite labor intensive and tiresome. There is, of course, a significant body of research about this and here is one example of it.

What is interesting in this article is that the authors use temporal data about the defect reports to assign teams. From the abstract: “In this article, we describe a new BA approach that relies on two key intuitions. Similar to traditional BA methods, our method constructs the expertise profile of project developers, based on the textual elements of the bugs they have fixed in the past; unlike traditional methods, however, our method considers only the programming keywords in these bug descriptions, relying on Stack Overflow as the vocabulary for these keywords. The second key intuition of our method is that recent expertise is more relevant than past expertise, which is why our method weighs the relevance of a developer’s expertise based on how recently they have fixed a bug with keywords similar to the bug at hand. “

The method uses text similarity measures to match defects and performs better than existing methods based on the meta-parameters. What it means in practice is that the only thing that is needed is the actual defect description, or actually a failure report in order to make the predictions.

Very interesting work to apply, it seems that the entry level is not that high for new companies.