Smoking is bad for you. Testing software is good for you though.

A while ago I wrote about detecting memory leaks in KDE CI. That works beautifully, and has already lead to real-world improvements, but it assumes that CI actually runs some code, usually a unit test.

While some KDE projects have fairly good test coverage, others do not so much. Particularly UI code isn’t very easy to usefully unit-test. More complex forms of testig like Selenium exist for UI testing, but doing it well is a significant investment and commitment from the developer.

That doesn’t mean there are no low-hanging fruits for more testing in KDE though. Many kinds of issues can be spotted by only running some piece of code, and checking for any abnormal signs. This kind of test is often known as smoke testing. Abnormal signs here would include things like crashes, QML errors, runtime warnings, or memory leaks. This is particularly useful when combined with debugging tools like asserts, address sanitizer, leak sanitizer etc.

A simple way to do smoke testing for applications is to use a command-line option for it that starts the application normally and automatically exists after some amount of time (e.g. 1 second). If the exit code is non-zero the test is considered failed. Alligator is an example of an application that has such a smoke test.

Applications are not the only smoke-testable thing though. We can also do it for systemsettings modules. Recently I added such a smoke test flag to kcmshell6, which can be used to smoke test KCMs. Doing that in a few projects already exposed some issues like memory leaks or missing dependency declarations, which since have been fixed.

Other things that could be tested in a similar way are Plasma applets (using e.g. plasmawindowed), runners, KIO workers, thumbnailers, or really anything that can be instantiated in a test. It could also be done for packaged binaries like Flatpaks, APKs, or distribution packages. Patches for this are most certainly welcome.