Sociotechnical Challenges of Machine Learning in Healthcare and Social Welfare
By: Tyler Reinmund, Lars Kunze, Marina Jirotka
Potential Business Impact:
Helps doctors use AI without messing up patient care.
Sociotechnical challenges of machine learning in healthcare and social welfare are mismatches between how a machine learning tool functions and the structure of care practices. While prior research has documented many such issues, existing accounts often attribute them either to designers' limited social understanding or to inherent technical constraints, offering limited support for systematic description and comparison across settings. In this paper, we present a framework for conceptualizing sociotechnical challenges of machine learning grounded in qualitative fieldwork, a review of longitudinal deployment studies, and co-design workshops with healthcare and social welfare practitioners. The framework comprises (1) a categorization of eleven sociotechnical challenges organized along an ML-enabled care pathway, and (2) a process-oriented account of the conditions through which these challenges emerge across design and use. By providing a parsimonious vocabulary and an explanatory lens focused on practice, this work supports more precise analysis of how machine learning tools function and malfunction within real-world care delivery.
Similar Papers
Designing Beyond Language: Sociotechnical Barriers in AI Health Technologies for Limited English Proficiency
Human-Computer Interaction
Helps doctors understand patients who don't speak English.
Sustainability of Machine Learning-Enabled Systems: The Machine Learning Practitioner's Perspective
Software Engineering
Helps build computer programs that are good for everyone.
Diachronic and synchronic variation in the performance of adaptive machine learning systems: The ethical challenges
Human-Computer Interaction
Keeps medical AI fair as it learns.