Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles

【类型】期刊

【作者】Salvini Pericle  Reinmund Tyler  Hardin Benjamin  Grieman Keri  Ten Holter Carolyn  Johnson Aaron  Kunze Lars  Winfield Alan  Jirotka Marina 

【刊名】Frontiers in Political Science

【关键词】 oversight ,Boeing,making,decision,MAX,social,predictive,for,care ,autonomous,monitoring,MCAS , ,driver,systems ,737,human,algorithm

【摘要】 This paper draws on three case studies to examine some of the challenges and tensions involved in the use of Autonomous Decision-Making Systems (ADMS). In particular, the paper highlights: (i) challenges around the shifting “locale” of the decision, and the associated consequences for stakeholders; (ii) potential implications for stakeholders from regulation such as the General Data Protection Regulation (GDPR); (iii) the different values that stakeholder groups bring to the “decision” question; (iv) how complex pre-existing webs of stakeholders and decision-making authorities may be disrupted or disempowered by the use of an automated system and the lack of evaluation of possible consequences; (v) how ADMS for non-technical users can lead to circumvention of the boundaries of intended system use. We illustrate these challenges through case studies in three domains: adult social care, aviation, and vehicle driver monitoring systems. The paper closes with recommendations for both practice and policy in the deployment of ADMS.

【年份】2023

【作者单位】Department of Computer Science, University of Oxford, Oxford, United Kingdom;Department of Engineering, Design and Mathematics, University of Bristol, Bristol, United Kingdom