Complex software and statistical modelling dominates all aspects of business and much of our daily lives. It is nearly impossible to confirm that computer programs running numerous statistical analyses, with multiple subroutines are generating the correct answers. Small errors in the code can cause problems that are very hard to spot. New and improved independent methods are needed to give both programmers and their customer’s confidence that software provides the correct result every time.

The EMRP project Traceability for computationally-intensive metrology (TraCIM) developed the TraCIM system - a new approach to software validation. This on-line system provides users with reference test data, and compares the results generated with those from an ‘ideal’ data set leading to a software validation test report. The project:

• Developed a web-based searchable database containing standardised mathematical language to specificy calculations needed for co-ordinate measurement machine software.
• Developed an assessment method for evaluating the performance of user software based on ‘ideal’ reference input data and corresponding ‘ideal’ reference results.
• Developed a method to combine multiple data set uncertainties to generate a single ‘figure of merit’ for the software under test. This can be used as an estimation of the software uncertainty component during the calculation of overall combined measurement uncertainties.

Confidence in the ongoing performance of software for QA and demonstrating that new software produces accurate results are important in many diverse applications from safety assessment of structures such as fuel storage tanks to quality control assessments of intricately shaped 3D components vital in precision engineering. Results generated by complex software can contain errors. Rounding approximations are assumed to be insignificant, or iterative analysis processes stop too soon failing to generate optimised convergence value are two sources of errors that are difficult for programmers to spot.

The project’s TraCIM system has been used to verify applications where safety critical or very extensive data sets require complex modelling and multistage computer programs to generate results. A follow on EMPIR Support for Impact project Validation of software development and analysis tools is extending this project’s TraCIM system for use with the popular Mathlab software tool. This will enable greater access to this software validation tool and broaden its uptake by industries with complex data and software manipulation requirements.

EMPIR project 15SIP06 ValTraC builds on this work.