Black box medicine and transparency
What ethical and legal rules should, and could, apply to black box medicine?
Research
Artificial intelligence is increasingly used throughout health systems, supporting healthcare professionals to make diagnoses through imaging and pathology and to optimise decisions about treatments and patient management.
Together with some complex algorithms, some of these uses are examples of ‘black box medicine’ in which opaque computational models are used to make decisions related to healthcare. These models can help speed up diagnoses as well as develop new treatments but, due to the amount of data they use and their complexity, their ‘reasoning’ cannot be explicitly understood or even stated.
The PHG Foundation has been awarded seed funding from the Wellcome Trust to examine what ethical and legal rules should, and could, apply to black box medicine. Clarifying the requirements for transparency and explanation could help to improve patient and public trust in how these technologies are used in healthcare.
This project consists of three phases that explore different aspects of transparency and explanation relating to the use of algorithms in healthcare:
Phase 1: We will undertake a philosophical evaluation of the ethical principles that apply to transparency in black box medicine
Phase 2: We will analyse what is legally required. In particular, we will explore the scope, extent, and impact of the general principle of transparency and right to explanation under the General Data Protection Regulation, and, where applicable, the UK Data Protection Act 2018. We will then assess competing explainable machine learning approaches to determine whether they meet these requirements
Phase 3: Using our knowledge from phases 1 and 2 of what is ethically and legally required, we aim to develop a tool to assist developers to think through the practical, ethical, and legal reasons to make their machine learning model human interpretable.
Outputs from this project will include reports, briefing materials and an academic paper.
For more details of this project please contact Alison Hall
