Smoke testing for machine learning: simple tests to discover severe bugs | SpringerLink
Machine learning systems are very popular today, at least when it comes to research applications. They are not as popular as one would wished (or liked) in the real applications. One of the reasons is the fact that they are hard to test. We do not know how to check if an algorithm will behave as expected in all similar situations – well, we do not know which situations are similar for us and for the ML system.
This paper looks at the problem from a different angle. The research question is: RQ: What are simple and generic software tests that are capable of finding bugs and improving the quality of machine learning algorithms?
The authors developed a set of smoke tests, which they see that all ML algorithms should pass. The paper is quite exhaustive and if you are interested, I recommend to take a look at this table:
Table 1 | Smoke testing for machine learning: simple tests to discover severe bugs | SpringerLink
I love the article. It is simple, to the point and very applied. I’m going to use that in my tests of ML algorithms in the future.
