SE metrics (Software Engineering) – Page 9 – Software engineering, metrics, functional safety …

What will the future bring ….

https://www.amazon.se/Brief-Answers-Big-Questions-Stephen/dp/1473695996/ref=sr_1_1?dchild=1&keywords=Hawking&qid=1625726909&sr=8-1

As my summer goes on, I’ve decided to take a look at the book of one of my favorite authors and scientists – Prof. Stephen Hawking. I’ve loved his books when I was younger and I like the way he could bring difficult theories to the masses, like his famous “gray holes” as opposed to the black ones.

This book allowed me to reflect on some of the most common questions that people ask, even to me. Like whether AI will take over or whether we should even invest in AI. Since I’m not a physicist, I cannot answer most of the questions, but I think the AI question is something that I can even attempt.

So, will AI take over? Is GPT-3 something to worry about? Will we be out of work as programmers? Well, not really. I think that we live in a world that is very diverse and that we need human judgement to make sure that we can live on. Take the recent cyber attack on Kaseya, which is a US-based company with thousands of clients. The attack affected a minority of their clients, some 40 or so (if I remember the article correctly). However, it make the entire COOP chain in Sweden stranded. Food was given away for free as there was no way to take payments. Other grocery shops bought the grocery stock from the affected chain, to make sure people have enough food. So, what would AI do?

Let’s think statistically for a moment. 40 customers, out of ca. 40,000, is about 0.1 percent. So, ignoring this event would just make 99.9% accuracy for AI. Is this good? Statistically, this is great! Almost perfect. For AI, therefore, this would be like a great way of optimizing. Not paying ransom, make sure that 0.1% is taken as a negligible error somewhere.

Now, let’s think about the social value. Without knowing the rest of the customers affected, or even the ones that were not affected, I could say that the value of having food on your table trumps many other kinds of value. Well, maybe not your health, but definitely something like a car or a computer game. So, the societal impact of that is large. We could model that in the AI, but there is programmatic problem. How to calculate value of diverse things – a car or food. There is the monetary value, of course, but it’s not constant in time. For someone who is hungry for days, the value of a sandwich is infinitely larger than the value of the same sandwich for someone who has just eaten a delicious stake. Another problem is that the value is dependent on the location (is there another grocery shop close by?), your stock (which is individual and hard to find for AI), or even the ability to use another system of payment (can I just get my groceries and pay later?)

This example shows an inherent problem in finding the right data to use for AI. I believe that this is a problem that will not really be solved. And if it cannot be solved, I think I would like to pay a few cents extra to have a human in the loop. I would liked to know that there is an option, in the event if a hacker attack, to talk to a person who understands my needs and can help me. Give me food without paying, knowing where I live and that I will pay later.

Until there are models which understand us, humans, we need to be able to stick to having humans in the loop. Given that there are ca. 6 billion people in the world, potentially different, with conflicting needs, I do not things AI will be able to help us in critical parts of the society.

Thinking machines – summer reading

Tänkande maskiner: Den artificiella intelligensens genombrott: Häggström, Olle, Pettersson, Rasmus: Amazon.se: Books

Summer is around the corner, in some places it has actually arrived. This usually makes for relaxation and time for reflection.

I would like to recommend a book from fellow professors. The book is about the way in which thinking machines are considered today and about the potentials of GAI, General AI. The book is written in a popular science way, but the examples and the research behind this is solid. The authors discuss both the technology, but also the society – legislative aspects of AI and understanding of what it means to be thinking in general, and in some specific cases tool.

Have a great summer and stay tuned for new posts after the summer.

New from ML?

I seldom write about films and events, well maybe actually never, but this year, a lot has happened in the online way.

What’s new in Machine Learning | Keynote – YouTube

The video above is about the news from Google about their TensorFlow library, which include new ways of training models, compression and performance tuning and more.

TensorFlow Light and TensorFlow JS allow us to use the same models as for desktops, but on mobile devices. Really impressive. I’ve caught myself thinking whether I’m more impressed by the hardware capabilities of small devices, or the capabilities of software. Either way – super cool.

Google is not the only company announcing something. NVidia is also showing a lot of cool features for enterprises. Cloud access for rapid prototyping, model testing and deployments are in the center of that.

NVIDIA Executive Keynote for Enterprise AI at COMPUTEX 2021 – YouTube

I like gaming, so this is impressing, but even more impressive is to look at the last-year’s DLSS technology, which still cannot be beaten by the competition. Really nice.

New programming tools?

Glinda: Supporting Data Science with Live Programming, GUIs and a Domain-specific Language (acm.org)

I’m not going to add a picture here, because the actual paper contains a great picture, which is copyrighted. But, do we need another tool (I though), and if you think like that… well, think again.

Once I looked at the paper, I really liked the idea. This is a tool that combines the programming tasks of software engineers and such tasks like data exploration, labelling or cleaning. It’s a kind of tool like Jupyter Notebook, but it allows to interact with the data in a deeper way.

I strongly recommend to take a look at the tool. I’ve done a quick check and it looks really nice.

Challenges when using ML for SE (article review)

104294.pdf (scitepress.org)

Machine learning has been used in software engineering for a while now. It used to be called advanced statistics, but with the popularization of artificial intelligence, we use the term machine learning more often. I’m one of those who like to use ML. It’s actually a mesmerizing experience when you train neural networks – change one parameter, wait a bit and see how the network performed, then again. Trust me, I’ve done it all too often.

I like this paper because it focuses on challenges for using ML, from the abstract:

” In the past few years, software engineering has increasingly automating several tasks, and machine learning tools and techniques are among the main used strategies to assist in this process. However, there are still challenges to be overcome so that software engineering projects can increasingly benefit from machine learning. In this paper, we seek to understand the main challenges faced by people who use machine learning to assist in their software engineering tasks. To identify these challenges, we conducted a Systematic Review in eight online search engines to identify papers that present the challenges they faced when using machine learning techniques and tools to execute software engineering tasks. Therefore, this research focuses on the classification and discussion of eight groups of challenges: data labeling, data inconsistency, data costs, data complexity, lack of data, non-transferable results, parameterization of the models, and quality of the models. Our results can be used by people who intend to start using machine learning in their software engineering projects to be aware of the main issues they can face. “

So, what are these challenges? Well, I’m not going to go into details about all of them, but I’d like to focus on the ones that are close to my heart – data labelling. The process of labelling, or tagging, data is usually very time consuming and very error-prone. You need to be able to remember how you actually labelled the previous data points (consistency), but also understand how to think when finding new cases. This paper does not list the challenges, but gives a pointer to a few paper where they are defined.

How much does a testing cost and how to estimate it…

CSUR5403-53 (acm.org)

Testing of software systems is a task which costs a lot. As a former tester, I see this as a neverending story – you’re done with your testing, new code is added, you are not done anymore, you test more, you’re done, new code…. and so on.

When I was a tester, there was no tools for automating the test process (we’re talking 1990’s here). Well, ok, there was CppUnit, and it was a great help – I could create a suite and execute it. Then I needed to add new test cases, create functional tests, etc. It was fun, until it wasn’t anymore.

I would have given a lot for having tools for test orchestration back then. A lot of things happened since then. This paper presents a great overview of how testing cost is estimated – I know, it’s not orchestration, but hear me out. I like this paper, because it shows anyways which tools are used, how test cost is estimated (e.g. based on metrics like coverage, effort, etc) and how the tests are evaluated.

I recommend this reading as an overview, a starting point for understanding the testing processes today, and, eventually, to optimize the test processes based on the right premises (not HiPPO).

If you want to test your CI test prioritization

Jin_Servant_ICSE21_AE.pdf (vt.edu)

Many of companies talk about using AI in their software engineering processes. However, they have problems in sharing their data with researchers and students. The legal processes with open sourcing data were and are scary. The processes of setting up internal collaborations is time consuming and therefore it needs more effort.

So, this is a great example of replicating some industrial set-ups in the open source community. I’ll use these data sets in my work and I’d love to see more initiatives like that.

Our team is working on one of those at the moment…

What makes a great code maintainer…

BIld av Rudy and Peter Skitterians från Pixabay

ICSE2021_B.pdf (igor.pro.br)

For many of us, software engineering is the possibility to create new projects, new products and cool services. We do that often, but we equally often forget about the maintenance. Well, maybe not forget, but we deliverately do not want to remember about it. It’s natural, as maintaining old code is not really anything interesting.

When reading this paper, I’ve realized that my view about the maintenance is a bit old. In my time in industry, maintainance was “bug-fixing” mostly. Today, this is more about community work. As the abstract of this paper says: “Although Open Source Software (OSS) maintainers devote a significant proportion of their work to coding tasks, great maintainers must excel in many other activities beyond coding. Maintainers should care about fostering a community, helping new members to find their place, while also saying “no” to patches that although are well-coded and well-tested, do not contribute to the goal of the project.”

This paper conducts a series of interviews with software maintainers. In short, their results are that great software maintainers are:

Available (response time),
Disciplined (follows the process),
Has a global view of what to achieve with the review,
Communicative,
Emapthetic,
Community building,
Technically excellent,
Quality aware,
Has domain experience,
Motivated,
Open minded,
Patient,
Diligent, and
Responsible

It’s a long list and the priority of each of these characteristics differs from one reviewer to another. However, it’s important that we see software maintainer as a social person who can contribute to the community rather than just sit in the dark office and reads code all day long. The maintainers are really the persons who make the software engineering groups work well.

After reading the paper, I’m more motivated to maintain the community of my students!

Siri, Write the Next Method… (article highlight)

Wen2021a.pdf (usi.ch)

I’ve came across this article by accident. Essentially I do not even remember what I was looking for, but that’s maybe not so important. Either way, I really want to try this tool.

This research study is about designing a tool for code completion, but not just a completion of a word/statement/variable, but providing a signature of the next method to implement.

From the abstract: “Code completion is one of the killer features of Integrated Development Environments (IDEs), and researchers have proposed different methods to improve its accuracy. While these techniques are valuable to speed up code writing, they are limited to recommendations related to the next few tokens a developer is likely to type given the current context. In the best case, they can recommend a few APIs that a developer is likely to use next. We present FeaRS, a novel retrieval-based approach that, given the current code a developer is writing in the IDE, can recommend the next complete method (i.e., signature and method body) that the developer is likely to implement. To do this, FeaRS exploits “implementation patterns” (i.e., groups of methods usually implemented within the same task) learned by mining thousands of open source projects. We instantiated our approach to the specific context of Android apps. A large-scale empirical evaluation we performed across more than 20k apps shows encouraging preliminary results, but also highlights future challenges to overcome.”

As far as I understand, this is a plug-in to android studio, so I will probably need to see if I can use it outside of this context. However, it seems to be very interesting….