News

Digital mammography

Technical report for the collection of clinical images and data when training and validating AI

The project

Image-based AI systems for disease detection are increasingly being developed, making it necessary to test if they are effective and trustworthy in heterogeneous clinical settings. Approved guidelines are also required to evaluate these products in real-world applications.

The exponential increase in healthcare data over the last decade, as well as the fast-paced technology developments, have resulted in promising novel AI approaches for diagnostic applications and risk prediction. However, the adoption of AI in clinical settings remains limited, mostly due to:

• limited quality data and interoperability across heterogeneous clinical centres and electronic health records;
• absence of robust validation procedures;
• distrust of predictions and decisions generated by AI systems;
• lack of harmonised government proposals and consensus guidelines on steps for their adoption.

Metrology Partnership project Developing a metrological framework for assessment of image-based Artificial Intelligence systems for disease detection (22HLT05, MAIBAI) is working to design an imaged-based AI framework to be used for diagnostic imaging in the field of breast cancer screening.

The necessary technological infrastructure to access information from a range of mammography databases is going to be implemented, together with a clear methodology to benchmark the quality of AI systems in heterogeneous clinical settings. In view of a standardized diagnostic framework to be applied in real-world scenarios across Europe, MAIBAI is developing:

- robust and safe procedures for medical image collection, sharing and custodianship;

- data augmentation techniques for solving domain-gap issues;

- quantitative performance metrics and methods for supporting explainability;

- protocols for the assessment of AI systems for disease screening, with a focus on understanding their generalisability and sensitivity to varying populations, manufacturers, image processing, and acquisition techniques.

The report

Artificial Intelligence is used for a variety of reasons in radiology and mammographic screening such as cancer detection, and triage. To be able to train, and validate AI products, it is vital to collect a range of clinical images encompassing the full patient population with good quality associated clinical data. However, there are many key challenges in collecting images, not only technical but also ethical.

To address these challenges, the project has published a report:

Technical specification of a framework for the collection of clinical images and data

The aim of this report, prepared by the Royal Surrey NHS Foundation Trust, is to describe the characteristics of a collection framework for radiological images for use when training and validating AI tools. This includes not only the collection of images and clinical data, but the ethics and information governance processes to consider ensuring the data is collected safely, and the infrastructure and agreements required to allow for the sharing of data.

The document is intended to provide a comprehensive outline of the purpose, operation, methods, policies and governance of collecting medical images for tuning, training, testing and/or evaluating AI products. Detailed instructions are given on the development of clinical image collection frameworks, enabling safe, automated and ongoing collection of clinical data (patient, diagnosis, outcomes) from multiple radiology sites, with a guarantee of data protection. The report is conceived as a good practice guide for end users (e.g. clinical database managers, screening data managers, researchers) involved in the management of clinical and image data, which could then be used for related AI applications.

Training videos available

The following training videos are available via the project’s website:

• Introduction to explainable AI;
• Metrology in AI;
• Image acquisition key-factors and image processing in mammography;
• OPTIMAM Mammography Image Database OMI-DB.

Project coordinator Alessandra Manzin from INRiM said

‘The standardised and impartial assessment framework developed in MAIBAI will enable more efficient, reliable, and reproducible validation of image-based AI systems for disease detection. This in turn will result in the scalability of AI systems for disease detection with a focus on breast cancer screening, enabling the reliable and safe use of AI in clinical settings. This could also address issues with delays in reading images due to highly-skilled staff shortages, which are expected to worsen over the next few years. To this purpose, MAIBAI is supporting the development of explainable and traceable AI tools for mammographic image analysis. It is also generating methods for data curation, data processing and data augmentation of the existing databases, providing validated methodologies to the medical community working in breast screening. It is expected that the utility of the methodologies developed within MAIBAI can also impact on other AI assessment tasks in healthcare across Europe.’

This Metrology Partnership project has received funding from the European Partnership on Metrology, co-financed by the European Union Horizon Europe Research and Innovation Programme and from the Participating States.

Want to hear more about EURAMET?

Sign up for EURAMET newsletters and other information

Follow us on LinkedIn and X