Evidence generation plan for artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway
6 Implementation considerations
The company should work with providers and central NHS England teams to begin the research. Planning for a prespecified period for the set-up of the technology is advised. During this period, training and implementation should be done before data collection is started, to account for learning effects. The following considerations around implementing the research process have been identified through working with system partners:
-
For safe implementation, developers could initially do 'silent evaluation' (see Kwong et al. 2022) before full deployment into services. This approach deploys the technology without any influence on clinical decision making until the technology is fully deployed. This approach can be used to understand whether the technology can be deployed safely (including in subpopulations), what the influence on decision making would likely have been (for example, onward referrals), and may collect some relevant data items (for example, test failure rate or number of indeterminate findings).
-
The company should provide training for staff in using the technology.
-
Services should be carefully selected to, when appropriate, maximise data collection for subgroups of interest.
-
To assess the potential for automation bias after deployment, the company may want to track the rate of diagnostic disagreement over time.
Potential barriers to implementation include:
-
the availability of research funds for data collection, analysis and reporting
-
the availability of NHS funding to cover the costs of implementing the technology in clinical practice
-
lack of expertise and staff to collect data
-
burden on clinical staff, including the need to have training ahead of implementation, data collection and follow up
-
differences in practice between primary care settings across the NHS
-
differences in skill and experience among staff when using the AI technology.
ISBN: 978-1-4731-6976-0
This page was last updated: