Author: Miroslaw Staron

I’m professor in Software Engineering at Computer Science and Engineering. I usually blog about interesting articles (for me) and my own reflections on the development of Software Engineering, AI, computer science and automotive software.

When it gets too much or Revenge of the Tipping point…

BIld av Katja S. Verhoeven från Pixabay

https://www.bokus.com/bok/9780316575805/revenge-of-the-tipping-point-overstories-superspreaders-and-the-rise-of-social-engineering/?utm_campaign=Performance%20Max%20%7C%20English%20%7C%20Rooth&gad_source=1&gclid=Cj0KCQiAuou6BhDhARIsAIfgrn4spmK1A21PF2Luov0HXzMwMFMsTcJKUSsvnIH5UEfxDs_lBz3TOUMaAuLEEALw_wcB

I’ve just finished reading this great book about the way in which the tipping point tips to the wrong side. It’s mostly about the law of “The large effect of the few” as Malcolm Gladwell puts it. In short, this law means that in certain situations, it’s the minority that is responsible for large effects. For example, the minority of old, badly maintained cars that contribute to to over 55% of pollution in one of the US cities. It’s about when one person, a superspreader, ends up in very specific conditions that allow this person to spread the contagion of the COVID virus at the beginning of the pandemic.

Now, we see that in software engineering a lot when we look at the tooling that we use. Let’s take the CI/CD tool Jenkins as an example. It is one of many different tools that were on the market at that time. It was not even the major one, but it was a sibling to a professional tool that was maintained by Oracle (if I recall correctly). Yet, it became very popular and the other tools did not. Since they were siblings, they were not worse, not better either; maybe a little different. What made it tip was the adoption of this tool in the community. A few superspreaders started to use it and discovered how good the tool is for automation of CI/CD tasks.

I see the same parallel to AI today. What was it that tipped the use of AI? IMHO it was a few things:

  1. Google’s LSTM use in Search – since there was a commercial value, it made sense to adopt it. Commercial adoption means business value, improvement and management focus (funding).
  2. Big data – after almost a decade of talking about big data, collecting it and indexing it, we were ready to provide the data-hungry modules with the data they needed to do something useful.
  3. HuggingFace – our ability to share models and use them without requirements on costly GPUs and large (and good) datasets.
  4. Access to competence – since we have so many skilled computer scientists and software engineers, it was easy to get hold of the competence needed to turn ideas into products. Google’s Deepmind is a perfect example of it. People behind it got the Nobel Prize.

Well, the rest is history as they say…. But, what will the next invention on the verge of the tipping point be?

Never eat alone, or else….

Image by Silviu on the street from Pixabay

Never Eat Alone: Keith Tahl Ferrazzi Raz: 9780241004951: Amazon.com: Books

In academia, the motto is “publish or perish”, with the emphasis on publishing. It’s for a good reason – we, academics, scholars, researchers, exist in a complex network of dependencies. We need others to get inspiration, understanding and when we get stuck.

If you look at the nobel prize winners, most of them work together. Listening to them I get an impression that you cannot become great by sitting in your own room and hatching ideas. But, at the same time, we are often introverts, at least I am.

This book is a great example of how we can build our networks and make meaningful connections. It helped me to realize how to be good at meaningful networking, not the one where you focus on meeting as many people as possible or as important people as possible. No, it’s about how to meet all kinds of people and how to learn from them. It’s about how to identify even a single item of information that you can use in your own work and for your own benefit.

I recommend this as a reading for one of those dark, autumn evenings that are inevitable coming now….

Materials that shape the world (of computing)

Material World: A Substantial Story of Our Past and Future: Ed Conway: 9780753559178: Amazon.com: Books

As a software engineer, I take hardware for granted. Moore’s law has taught me that all kind of computing power grows. My experience has taught me that all computing power is then consumed by frameworks, clouds and eventually is not enough.

This great book shades a really interesting light on the way in which materials like Lithium and Silicon shape our society. We think that TSMC is one of the isolated companies that excelled in chip-making. The reality is that this company is great, but it is also only one in a long chain of suppliers of the chip industry. We learn that the sand which is used to make chips comes from the US, not from Taiwan. We learn that the lithium used in our batteries comes often from the Andes, Chile, not from China. We also learn that the ONLY way for the humanity to progress is to collaborate cross-nations. If we don’t do that, no single country in the world has the machinery, the know-how and the competence to develop our modern technology.

It is in a series of great readings for software engineers when they start their studies today.

How innovations spread…

The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution: Isaacson, Walter: 9781476708706: Amazon.com: Books

Although this book is a bit dated – in the sense when we call everything that is pre-pandemic as dated – it is a great reading. It takes us on a journey of inventions in Silicon Valley, although it starts with Ada Byron and her work on computing machines.

I recommend this book because it goes against established theories in academia about innovations – that we innovate individually or in teams. Instead, it takes us on the journey of connections, research and innovation building on one another. It tells a great story how world’s technology evolves by taking one innovation and making another one. It is a story of global collaborations and how these collaborations are entangled with one another and support one another.

If it was up to me, this would be a mandatory reading for all new students of the university software engineering programs.

How to create a tribe…

Conspiracy: A History of Boll*cks Theories, and How Not to Fall for Them: Phillips, Tom, Elledge, Jonn: 9781472283405: Amazon.com: Books

During the holidays, I’ve had some time to read books outside of software engineering. This one caught my attention, not because I like conspiracy theories, but mostly because I got interested in how theories actually spread.

Now, while this book is about conspiracies, don’t get me wrong, it is also about creating communities and making things spread. Whether this is a conspiracy or maybe just a normal culture, it is interesting to read.

Anyways, if you have some time on your hands, instead of scrolling Instagram or Facebook feeds, take a look at this book. I guarantee that it will provide even more fun.

How to improve the code reviews

https://dl.acm.org/doi/pdf/10.1145/3660806

I have written a lot about code reviews. Not because this is my favourite activity, but because I think that there is a need for improvement there. Reading others’ code is not as fun as we may think and therefore making it a bit more interesting is much desirable.

This paper caught my attention because of the practical focus of it. In particular, the abstract caught my attention – they claim that changing the order of files presented for review makes a lot of difference. Up to 23% more comments are written when the files are arranged in the right order. Not only that, the quality of the comments seems to increase too. More tips:

  1. Re-order Files Based on Hot-Spot Prediction: The study found that reordering the files changed by a patch to prioritize hot-spots—files that are likely to require comments or revisions—improves the quality of code reviews. Implementing a system that automatically reorders files based on predicted hot-spots could make the review process more efficient, as it leads to more targeted comments and a better focus on critical areas.
  2. Focus on Size-Based Features: The study highlighted that size-based features (like the number of lines added or removed) are the most important when predicting review activities. Emphasizing these features when prioritizing files or creating models for review could further streamline the process.
  3. Utilize Large Language Models (LLMs): LLMs, such as those used for encoding text, have shown potential in capturing the essence of code changes more effectively than simpler models like Bag-of-Words. Incorporating LLMs into the review tools could improve the detection of complex or nuanced issues in the code.
  4. Automate Hot-Spot Detection and Highlighting: The positive impact of automatically identifying and prioritizing hot-spots suggests that integrating such automation into existing code review tools could significantly enhance the efficiency and effectiveness of the review process.

Sounds like this is one of the examples where we can see the large benefits of using LLMs in code reviews. I hope that this will make it into more companies than Ubisoft (partner on that paper).

I’ve asked ChatGPT to provide me an example of how to create such a hotspot model and it seems that this can be implemented in practice very easily. I will not paste it here, but please try for yourself.

Commander programmer and his army of bots… – Software Engineering in a post-GenAI era

Engineering software today becomes a profession where AI can make the most difference. GitHub and the availability of open source code, as well as natural language texts, provided the best possible fuel for creating large and great models. Some predict that we will not need programmers any more, but I predict that we will need them. I also think that these programmers will be much better than they are today. I view the future as quite optimistic and bright for our profession.

To begin with, I see the profession to change in a way that one single programmer will be more like a team leader for a number of LLMs or bots – hence the title. The programmer will use one model/bot for extracting requirements from standards, documents, twitters or other text sources. The requirement bot will help the programmer to find the user stories/requirements, to prioritize them and to set up a backlog.

Then, the programmer will use another bot to create high-level design. The bot will provide an initial design and will provide the programmer with some sort of conversational interface to reason about the design – the patterns, the architectural styles and the typical non-functional characteristics to maintain for the software.

At the same time, the programmer will use either the same bot or another one to write the source code of the software. We already use tools like GitHub CoPilot for these tasks and these tools will be even better. When constructing the software, the programmer will also use testing bots to create test cases and to improve the programs. Here, the work on program repair is definitely very interesting as it provides the ability to automatically improve the low-level design of the software.

Finally, when the software is complete, then during the release, the programmer will use bots to monitor the software. Finding defects fast, monitoring the performance, availability and security of software will be delegated to even more bots. They will do the work faster than any programmer anyways.

The job of the programmer will then be to instruct, integrate and monitor these bots. A little bit like a team leader who needs to make sure that all team members agree on certain principles. This means that the programmers will need to know more about the domains where they work as well as they will need access to tooling that supports their workflows.

What will also change is our perception of quality of software. If we can use these bots and make the software construction much faster, then we probably will not need to write super-maintainable code – the bots will be able to decipher even the most obscure code and help us to improve it. Hey, maybe these bots will even write a completely new version of our software once they learn how to optimize their design and when we learn how to link these bots to one another. Regardless, I do not think they will be able to write a Skynet kind of code….

On Security Weaknesses and Vulnerabilities inDeep Learning Systems

Image: wordcloud based on the text of the article

In the rapidly evolving artificial intelligence (AI), deep learning (DL) has become a cornerstone, driving advancements based on transformers and diffusers. However, the security of AI-enabled systems, particularly those utilizing deep learning techniques, are still questioned.

The authors conducted an extensive study, analyzing 3,049 vulnerabilities from the Common Vulnerabilities and Exposures (CVE) database and other sources. They employed a two-stream data analysis framework to identify patterns and understand the nature of these vulnerabilities. Their findings reveal that the decentralized and fragmented nature of DL frameworks contributes significantly to the security challenges.

The empirical study uncovered several patterns in DL vulnerabilities. Many issues stem from improper input validation, insecure dependencies, and inadequate security configurations. Additionally, the complexity of DL models makes it harder to apply conventional security measures effectively. The decentralized development environment further exacerbates these issues, as it leads to inconsistent security practices and fragmented responsibility.

It does make sense then to put a bit of effort into securing such systems. By the end of the day, input validation is no rocket science.

Federated learning in code summarization…

3661167.3661210 (acm.org)

So far, we have explored two different kinds of code summarization – either using a pre-trained model or training our own. However, both of them have severe limitations. The pre-trained models are often good, but too generic for the project at hand. The private models are good, but often require a lot of good data and processing power. In this article, the authors propose to use a third way – federated learning.

The results show that:

  • Fine-tuning LLMs with few parameters significantly improved code summarization capabilities. LoRA fine-tuning on 0.062% of parameters showed substantial performance gains in metrics like C-BLEU, METEOR, and ROUGE-L.
  • The federated model matched the performance of the centrally trained model within two federated rounds, indicating the viability of the federated approach for code summarization tasks.
  • The federated model achieved optimal performance at round 7, demonstrating that federated learning can be an effective method for training LLMs.
  • Federated fine-tuning on modest hardware (40GB GPU RAM) was feasible and efficient, with manageable run-times and memory consumption.

I need to take a look at this model a bit more since I like this idea. Maybe this is the beginning of the personalized bot-team that I always dreamt of?

On the Use of ChatGPT for Code Review (article review)

3661167.3661183 (acm.org)

CoPilot and other tools have been used increasingly often, but mostly for testing and for programming. Now, the questions is whether these kind of tools help much in such tasks as code review.

This paper explores how developers utilize ChatGPT in the code review process and their reactions to the AI-generated feedback. This research analyzed 229 review comments from 205 pull requests across 179 projects to understand the purposes and effectiveness of ChatGPT in code review.

The found that:

  • Developers primarily use ChatGPT for two main purposes: referencing and outsourcing. These purposes were further categorized into several sub-categories:
  • Developers used ChatGPT to gain understanding and support their opinions, including tasks such as refactoring, implementation, design, non-programming tasks, testing, documentation, and others.
  • Developers directly asked ChatGPT to resolve specific issues. This included implementation, refactoring, bug-fixing, reviewing, testing, design, documentation, and other tasks.

The study found a mixed reaction to ChatGPT-generated reviews, which is not really surprised given that it is a new technology and code reviews are not only for review, but also for learning:

  • Positive Reactions (64%): A majority of the responses were positive, indicating that developers found the AI’s suggestions helpful.
  • Negative Reactions (30.7%): A significant portion of responses were negative. The primary reasons for dissatisfaction included the solutions not bringing extra benefits, containing bugs, or not aligning with developers’ preferred coding styles.

However, what I find really interesting.

  • Enhanced Prompt Strategies: Effective use of ChatGPT requires well-crafted prompts to maximize the AI’s potential in generating useful code reviews.
  • Tool Integration: Integrating ChatGPT with existing development tools can streamline the review process.
  • Continuous Monitoring: Regular assessment and refinement of ChatGPT’s outputs are necessary to ensure high-quality code reviews.

These three points are kind of cool, because they mean that we need to learn how to instruct and use these tools. That means that we loose something (knowledge about our products), while we need to learn more general skills about prompting…