Between Knowledge and Care: Evaluating Generative AI-Based IUI in Type 2 Diabetes Management Through Patient and Physician Perspectives
By: Yibo Meng , Ruiqi Chen , Zhiming Liu and more
Potential Business Impact:
AI gives health advice, but doctors say it's not safe.
Generative AI systems are increasingly adopted by patients seeking everyday health guidance, yet their reliability and clinical appropriateness remain uncertain. Taking Type 2 Diabetes Mellitus (T2DM) as a representative chronic condition, this paper presents a two-part mixed-methods study that examines how patients and physicians in China evaluate the quality and usability of AI-generated health information. Study~1 analyzes 784 authentic patient questions to identify seven core categories of informational needs and five evaluation dimensions -- \textit{Accuracy, Safety, Clarity, Integrity}, and \textit{Action Orientation}. Study~2 involves seven endocrinologists who assess responses from four mainstream AI models across these dimensions. Quantitative and qualitative findings reveal consistent strengths in factual and lifestyle guidance but significant weaknesses in medication interpretation, contextual reasoning, and empathy. Patients view AI as an accessible ``pre-visit educator,'' whereas clinicians highlight its lack of clinical safety and personalization. Together, the findings inform design implications for interactive health systems, advocating for multi-model orchestration, risk-aware fallback mechanisms, and emotionally attuned communication to ensure trustworthy AI assistance in chronic disease care.
Similar Papers
Design and Evaluation of an AI-DrivenPersonalized Mobile App to Provide MultifacetedHealth Support for Type 2 Diabetes Patients inChina
Human-Computer Interaction
Helps diabetes patients talk to doctors better.
Generative AI for Healthcare: Fundamentals, Challenges, and Perspectives
Artificial Intelligence
Makes AI better at helping doctors and patients.
From Symptoms to Systems: An Expert-Guided Approach to Understanding Risks of Generative AI for Eating Disorders
Human-Computer Interaction
AI can accidentally help people with eating disorders.