Using Clinical Guidelines, domain ontology, and LLMs for Personalized Leukemia Treatment Recommendations

Large Language Models (LLMs) offer new opportunities for clinical decision support, but face challenges in reliability, precise recommendations for individual patients, and adherence to medical guidelines. Challenges such as insufficient domain knowledge, generic outputs, and hallucination are risks to their clinical adoption. This paper proposes an approach that integrates LLMs with Clinical Practice Guidelines (CPGs) and medical ontologies to enhance personalized treatment recommendations. We compared four strategies to generate treatment recommendations with and without integrating clinical guidelines: (1) LLMs without any guideline input, (2) providing the full guideline document as textual input to LLMs with retrieval-augmented generation (RAG) technique; (3) converting guideline documents from PDF to markdown files capturing the structure of tables, diagrams, and references and using Chain-of-Thoughts to reason each decision steps; (4) structuring guidelines as graphs and linking medical concepts to ontologies as input to LLMs. We experimented on GPT-3.5 Turbo, GPT-4, and Llama 2. The evaluations assessed guideline adherence, treatment completeness, path alignment, and answer relevancy with Acute Lymphoblastic Leukemia as the primary use case. Additionally, we developed a user interface for health professionals to input patient descriptions and obtain treatment recommendations and explanations. Preliminary results demonstrate the feasibility of the graph-based approach in decision path tracing, graph-augmented reasoning, and natural language explanations to enhance transparency for clinician validation.