The possibility of AI-induced medical manslaughter: Unexplainable decisions, epistemic vices, and a new dimension of moral luck

The use of artificial intelligence (AI) systems in healthcare provides a compelling case for a re-examination of ‘gross negligence’ as the basis for criminal liability. AI is a smart agency, often using self-learning architectures, with the capacity to make autonomous decisions. Healthcare practitioners (HCPs) will remain responsible for validating AI recommendations but will have to contend with challenges such as automation bias, the unexplainable nature of AI decisions, and an epistemic dilemma when clinicians and systems disagree. AI decisions are the result of long chains of sociotechnical complexity with the capacity for undetectable errors to be baked into systems, which introduces a new dimension of moral luck. The ‘advisory’ nature of AI decisions constructs a legal fiction, which may leave HCPs unjustly exposed to the legal and moral consequences when systems fail. On balance, these novel challenges point towards a legal test of subjective recklessness as the better option: it is practically necessary; falls within the historic range of the offence; and offers clarity, coherence, and a welcome reconnection with ethics.