Title Explainable deep-learning-based diagnosis of Alzheimer’s disease using multimodal input fusion of PET and MRI images /
Authors Odusami, Modupe ; Maskeliūnas, Rytis ; Damaševičius, Robertas ; Misra, Sanjay
DOI 10.1007/s40846-023-00801-3
Full Text Download
Is Part of Journal of medical and biological engineering.. Heidelberg : Springer. 2023, vol. 43, iss. 3, p. 291-302.. ISSN 1609-0985. eISSN 2199-4757
Keywords [eng] Alzheimer’s disease ; deep learning ; feature fusion ; heuristic methods ; MRI ; PET
Abstract [eng] Purpose: Alzheimer’s disease (AD) is a progressive, incurable human brain illness that impairs reasoning and retention as well as recall. Detecting AD in its preliminary stages before clinical manifestations is crucial for timely treatment. Magnetic Resonance Imaging (MRI) provides valuable insights into brain abnormalities by measuring the decrease in brain volume expressly in the mesial temporal cortex and other regions of the brain, while Positron Emission Tomography (PET) measures the decrease of glucose concentration in the temporoparietal association cortex. When these data are combined, the performance of AD diagnostic methods could be improved. However, these data are heterogeneous and there is a need for an effective model that will harness the information from both data for the accurate prediction of AD. Methods: To this end, we present a novel heuristic early feature fusion framework that performs the concatenation of PET and MRI images, while a modified Resnet18 deep learning architecture is trained simultaneously on the two datasets. The innovative 3-in-channel approach is used to learn the most descriptive features of fused PET and MRI images for effective binary classification of AD. Results: The experimental results show that the proposed model achieved a classification accuracy of 73.90% on the ADNI database. Then, we provide an Explainable Artificial Intelligence (XAI) model, allowing us to explain the results. Conclusion: Our proposed model could learn latent representations of multimodal data even in the presence of heterogeneity data; hence, the proposed model partially solved the issue with the heterogeneity of the MRI and PET data.
Published Heidelberg : Springer
Type Journal article
Language English
Publication date 2023
CC license CC license description