lecture: Can we write perfect tests? - Maybe!

Event_large

The pros of a solid unit test suite are well understood and accepted in the ruby community. The problem: How to define solid? Traditional metrics like line-coverage, branch-coverage and even statement-coverage can be misleading. Having a statement executed once does not mean all edge cases are specified and bug-free! Automated tools can be used to identify uncovered edge cases that will introduce bugs into your program. Mutation testing brings fuzzing to the implementation level. Unlike input fuzzing it modifies the implementation to check if the test suite can detect a huge set of automatically introduced behaviour changes. This talk will elaborate the history of testing and the metrics that are used to define coverage. And how such metrics can and have misguided development direction. It will also show examples of projects that heavily adopted mutation testing. Especially the long term effects and how the (mutation)-metrics driven approach improved developer happiness and code stability. Showing what kind of actual bugs where caught and how code became naturally streamlined. Lastly the current and future limits of existing mutation testing tools will be presented.