A Respiratory Disease Management Framework by Combining Large Language Models and Convolutional Neural Networks for Effective Diagnosis

Abstract

Artificial Intelligence in medical diagnostics has the potential to significantly increase patient care and healthcare outcomes. This synergy between advanced artificial intelligence technologies not only optimizes the efficiency of diagnostic analysis but also holds significant promise in improving patient outcomes and supporting healthcare professionals in delivering precise medical interventions. This paper presents a radically new approach that combines the effect of Large Language Models (LLM) with Computer Vision techniques aimed at increasing the performance of medical diagnosis and treatment recommendations for respiratory diseases. For image analysis, we use a pre-trained LLM from Hugging Face, ‘Llama-2-7B-chat-GGML’, and Convolution Neural Networks (CNN) which consists of InceptionV3, MobileNetV2, and NASNet. The CNN was able to classify chest X-ray images to be 92.85%, 91.88%, and 95.92%. Moreover, the LLM is used to analyze clinical data and generate therapeutic recommendations. We achieved a reduction in inference time of around 33.1% from 165.6 seconds to 111.9 seconds in the most general scenario. Such interaction of CNN and LLM in system use increases the information value of medical diagnostic analysis with high potential for increasing the healthcare outcome. Detailed information about the workflow, diagnostic techniques, and recommendation generation is presented. Experimental analysis of the developed system indicates the application of a combination of LLM and CNN for medical diagnostic purposes to aid healthcare professionals in making informed decisions and providing precise medical advice.