How to measure software architectures?

The question of what to measure pops up very often in our work. It’s not easy to answer, but we give guidance for areas of software engineering. In this blog post, I focus on the area of software architectures.

For the software architectures, there are a few areas that are important:

  • Stability of the architecture
  • Complexity of the architecture
  • and Quality of the architecture

These areas can be quantified with measures of – interface stability, component complexity, technical debt and similar. Based on our work with the companies in Software Center, we realized that these measures are more important than more complex measures like – architecture weight, architecture stability index or architectural debt. The reason is that the simple measures provide an immediate understanding of what can be done to improve the situation.

In our work, we also looked at how to calculate these measures based on open data sets of complexity of automotive software.

If you are interested in the details of how to use the measures, please take a look at Chapter 7 of my book:

The beginning of your measurement program – the model

Software development measurement programs can take many shapes and forms. They are often aligned with the goals of their organizations. In this blog post I would like to provide a short summary of the most common measurement models which can be encountered in the field of software engineering today. The set of models is naturally growing all the time, so new suggestions are more than welcome.

To put this blog post in context, let’s start from the ISO 15939 standard, which is the official standard for measurement programs in the software and systems engineering industry. The standard provides both the vocabulary and the recommended hierarchical model of information needs, indicators and measures. The strength of this model is its unification with the othe ISO standards, like the newest ISO 25000 series, and in particular the ISO 2502x parts, i.e., quality measurement division.

However, ISO 15939 is not the only standard in this field, there are others too. Some of them are more well known and some are less, but most of the standards in my list are used somewhere.

  • GQM – the classical Goal Question Metric model, which is very flexible and can be used for almost any measurement program and purpose. It’s structure is similar to the structure of the measurement information model of ISO 15939, but it’s does not contain such concepts as information need and indicator.
  • GQIM – the extension of the classical model to include the notion of the indicator.
  • Lightweight GQM, http://publica.fraunhofer.de/eprints/urn_nbn_de_0011-n-212218.pdf – is a version of GQM for small and medium enterprises. The lightweight keyword here means that
  • MIS-PyME MCMM, http://www.sciencedirect.com/science/article/pii/S0965997810000682 –  a measurement maturity model, constructed in a similar manner to CMMI, but aligned with measurement needs and only about the measurement programs. To some extent, this model provides examples of measures to be used at different levels and in different areas.
  • SQIP – a model based on the CMMI requirements for the measurement programs in more mature organizations, from level 2 upwards (if I remember correctly).

And this is not the end of the list, we have more, like:

  • MESOPYME
  • QFD
  • AAHA
  • LQIM
  • HSC
  • GQM-DSFMS
  • Tarc
  • OMSD
  • SPGQM
  • SCAPT,
  • …. and some more…

These other approaches seem to be a bit more niche in both their adoption in industry and in academia. If you know, use or heard of, any other approaches, please let me know.