MAIBAI

Developing a Metrological framework for Assessment of Image-Based Artificial Intelligence systems for disease detection

https://www.maibaiproject.eu/

Before Artificial Intelligence (AI) can be introduced into imaging then its use needs to be understood and evaluated.  The MAIBAI consortium is formed of 12 European groups, mostly Metrology groups, its overall objective of MAIBAI is to develop a metrological framework necessary to support standardisation in image-based AI systems for disease detection. Using breast screening as an exemplar, the performance of explainable and traceable AI tools will be analysed on a large real-world database of mammographic images, informing the design of the standardised assessment framework.

The specific objectives of the project are:

1. To develop technical infrastructure to be able to query and extract the relevant data from various databases.
2. To identify the key factors differentiating clinically relevant subgroups in the data and use these to categorise the data into subsets based on these key factors, identifying where there is sufficient data for training and validation. To develop a methodology to generate data derived from physics-informed models based on measurement knowledge.
3. To use an explainable and traceable AI tool for disease screening, providing the capability to train and retrain the tool as necessary, to evaluate its performance under different scenarios, including low versus high image quality data, validation for specific patient demographics, etc. To develop and validate methods for the interpretation of the behaviour of the trained AI tool.
4. To summarise the performance testing evaluations and to provide metrology recommendations for the assessment of AI tools for disease screening- with a focus on understanding their generalisability and sensitivity to varying populations, manufacturers, image processing and acquisition techniques. To use the recommendations to design a global, standardised, and impartial AI assessment framework.
5. To facilitate the take up of the technology and measurement infrastructure developed in the project by the measurement supply chain, standards developing organisations, and end users (e.g. clinical stakeholders, manufacturers of medical and healthcare products, regulators).

Funding: The project 22HLT05 MAIBAI, started in September 2023, has received funding from the European Partnership on Metrology, co-financed from the European Union’s Horizon Europe Research and Innovation Programme and by the Participating States. Funding for the UK partners (including Royal Surrey) is provided by Innovate UK under the Horizon Europe Guarantee Extension.