Application of ChatGPT to Support Nutritional Recommendations for Dialysis Patients - A Qualitative and Quantitative Evaluation

Objectives: The rising diversity of food preferences and the desire to provide better personalized care provide challenges to renal dietitians working in dialysis clinics. To address this situation, we explored the use of a large language model, specifically, ChatGPT using the GPT-4 model (openai.com), to support nutritional advice given to dialysis patients. Methods: We tasked ChatGPT-4 with generating a personalized daily meal plan, including nutritional information. Virtual “patients” were generated through Monte Carlo simulation; data from a randomly selected virtual patient were presented to ChatGPT. We provided to ChatGPT patient demographics, food preferences, laboratory data, clinical characteristics, and available budget, to generate a one-day sample menu with recipes and nutritional analyses. The resulting daily recipe recommendations, cooking instructions, and nutritional analyses were reviewed and rated on a five-point Likert scale by an experienced renal dietitian. In addition, the generated content was rated by a renal dietitian and compared with a U. S. Department of Agriculture–approved nutrient analysis software. ChatGPT also analyzed nutrition information of two recipes published online. We also requested a translation of the output into Spanish, Mandarin, Hungarian, German, and Dutch. Results: ChatGPT generated a daily menu with five recipes. The renal dietitian rated the recipes at 3 (3, 3) [median (Q1, Q3)], the cooking instructions at 5 (5,5), and the nutritional analysis at 2 (2, 2) on the five-point Likert scale. ChatGPT's nutritional analysis underestimated calories by 36% (95% CI: 44-88%), protein by 28% (25-167%), fat 48% (29-81%), phosphorus 54% (15-102%), potassium 49% (40-68%), and sodium 53% (14-139%). The nutritional analysis of online available recipes differed only by 0 to 35%. The translations were rated as reliable by native speakers (4 on the five-point Likert scale). Conclusion: While ChatGPT-4 shows promise in providing personalized nutritional guidance for diverse dialysis patients, improvements are necessary. This study highlights the importance of thorough qualitative and quantitative evaluation of artificial intelligence–generated content, especially regarding medical use cases.