USING FAULT CHARACTERISTICS TO IMPROVE SOFTWARE FAULT PREDICTION

SIGNIFICANCE: Faults in software code are a significant cost to companies, as well as a risk to human safety and business success. Finding and fixing faults in code costs the UK software industry billions of pounds every year. Significant cost savings are available with even small improvements in our capability to find faults before systems are delivered to users. BACKGROUND: Our previous work shows that during the last 10 years, 208 studies have published hundreds of different fault prediction models. These studies are usually typified by researchers applying one or more of the many modeling techniques to one or more of the many available data sets, then applying performance measures to report how well that model predicts faults. PROBLEM: Models do not perform consistently above the current predictive performance ceiling of about 80% recall. We propose that an important contributor to this underperformance is that models treat all faults as homogeneous. No previous attempt has been made to understand what characteristics make a fault predictable or what features a model needs in order to predict faults with particular characteristics. AIM: To build a fault prediction model ensemble which is focused on the characteristics of faults and which consistently performs above the current performance ceiling. METHOD: This 36 month project is based on analysing the code and fault data from six commercial systems and from six open source systems. We will conduct detailed quantitative and qualitative analysis of the characteristics of the faults in these systems, identifying for example whether the characteristics of faults are problems in code interfaces, algorithmic problems, structural problems, typographic problems, etc. We will construct a set of prediction models with a large variety of features (e.g. different modeling techniques, different independent variables, etc.). We will use these models to empirically identify relationships between fault characteristics and the features of individual models. This means that we will identify what features of prediction models predict faults with particular characteristics. We will build ensembles of models with features that cover the widest range of fault characteristics. We will evaluate those models on industrial systems in collaboration with a company.