Predicting metamorphic relations: an evaluation of program representations and machine learning techniques

Abstract

Testing complex scientific applications can often be a complicated and expensive procedure. A test oracle is used to verify the behavior of the software under test. However, difficulties due to the implementation of a test oracle make the process of systematically testing scientific applications more challenging. This problem is known as the oracle problem. Metamorphic testing (MT) is an effective technique to test these applications as it uses metamorphic relations (MRs) to determine whether test cases have passed or failed. Metamorphic relations are essential components of metamorphic testing that highly affect its fault detection effectiveness. MRs are usually identified with the help of a domain expert, which is a labor-intensive task. In this work, a previously developed graph kernel-based machine learning method is extended by predicting MRs for functions that perform matrix calculations. Then, semi-supervised support vector machine (S3VM) is used to build the predictive model for the suggested approach. Finally, call graph (CG) information of the functions are used to calculate the graph kernels to predict MRs. The overall result shows that random walk kernel performs better than the graphlet kernel, and semi-supervised learning can be effective with more unlabelled data. Also, the use of call graph representation presents a new avenue of research in predicting MRs for unseen functions.