{"id":435,"date":"2020-05-11T08:38:06","date_gmt":"2020-05-11T08:38:06","guid":{"rendered":"https:\/\/metrics.blogg.gu.se\/?p=435"},"modified":"2020-04-26T10:04:24","modified_gmt":"2020-04-26T10:04:24","slug":"how-do-we-know-if-something-is-popular","status":"publish","type":"post","link":"https:\/\/metrics.blogg.gu.se\/?p=435","title":{"rendered":"How do we know if something is popular…"},"content":{"rendered":"\n

Investigating diversity and impact of the popularity metrics for ranking software packages (review): https:\/\/onlinelibrary-wiley-com.ezproxy.ub.gu.se\/doi\/pdfdirect\/10.1002\/smr.2265<\/a> <\/p>\n\n\n\n

\"\"
Image from Pixabay<\/figcaption><\/figure>\n\n\n\n

I’ve written about the ways of assessing how good software is. One of the modern approaches, which I talked about before, is the use of A\/B testing and online experiments. Providing the users with different versions of the features\/systems\/use cases allows the company to understand which of the options provides the best response from the users. <\/p>\n\n\n\n

However, there are a number of challenges with this approach – the most prominent being the potential existence of confounding factors. Even if the results show a positive\/negative response, we do not really know whether the response is not caused by something else (for example by users being tired, changes in the environment, etc.)<\/p>\n\n\n\n

After using GitHub, both as a user and as a researcher, I sometimes wondered whether the star system is actually the right one. I wondered whether we should use a sort-of A\/B testing system where we could check how often people usually access certain repositories. <\/p>\n\n\n\n

In this paper, the authors take a look at different ways of assessing popularity of repositories. The results show that regardless of the metrics, the popular repositories are popular – i.e. popularity is not dependent of a metric. <\/p>\n\n\n\n

Popularity metrics studied:<\/p>\n\n\n\n