| Title |
Adaptive vision–language transformer for multimodal CNS tumor diagnosis |
| Authors |
Nasir, Inzamam Mashood ; Alshaya, Hend ; Tehsin, Sara ; Bouchelligua, Wided |
| DOI |
10.3390/biomedicines13122864 |
| Full Text |
|
| Is Part of |
Biomedicines.. Basel : MDPI. 2025, vol. 13, iss. 12, art. no. 2864, p. 1-26.. ISSN 2227-9059 |
| Keywords [eng] |
CNS tumors ; domain generalization ; MRI diagnosis ; multimodal learning ; Vision–Language Transformer |
| Abstract [eng] |
Objectives: Correctly identifying Central Nervous System (CNS) tumors through MRI is complicated by utilization of divergent MRI acquisition protocols, unequal tumor morphology, and a difficulty in systematically combining imaging with clinical information. This study presents the Adaptive Vision–Language Transformer (AVLT), a multimodal diagnostic infrastructure designed to integrate multi-sequence MRI with clinical descriptions while improving robustness and interpretability to domain shifts. Methods: AVLT integrates the MRI sequence ((Formula presented.), (Formula presented.), (Formula presented.), FLAIR) and clinical note text in a joint process using normalized cross-attention to establish association of visual patch embeddings with clinical token representations. An Adaptive Normalization Module (ANM) functions to mitigate distribution shift across datasets by adapting the statistics of domain-specific features. Auxiliary semantic and alignment losses were incorporated to enhance stability of multimodal fusion. Results: On all datasets, AVLT provided superior classification accuracy relative to CNN-, transformer-, radiogenomic-, and multimodal fusion-based models. The AVLT model accuracy was 84.6% on BraTS (OS), 92.4% on TCGA-GBM/LGG, 89.5% on REMBRANDT, and 90.8% on GLASS. AvLT AUC values are at least above 90 for all domains. Conclusions: AVLT provides a reliable, generalizable, and clinically interpretable method for accurate diagnosis of CNS tumors. |
| Published |
Basel : MDPI |
| Type |
Journal article |
| Language |
English |
| Publication date |
2025 |
| CC license |
|
