Availability-Aware Multimodal Deep Learning for Breast Cancer Diagnosis with Missing Modalities

Abstract: The diagnosis of breast cancer frequently gains from multimodal imaging; however, comprehensive multimodal data are commonly inaccessible in standard clinical practice due to workflow challenges, budget restrictions, and uneven access to imaging resources. Many current multimodal deep learning models have restricted clinical use since they presume that all imaging modalities are accessible during inference. To overcome this constraint, we introduce a multimodal deep learning framework that is aware of missing modalities, incorporating modality availability modeling alongside reliability-informed decision support for breast cancer detection. The system utilizes modality-specific ResNet- 101 encoders for both mammography and ultrasound, along with a fusion module that is aware of availability and that dynamically modifies the contribution of each modality based on its presence. A parallel reliability estimation head forecasts diagnostic confidence, allowing uncertainty-informed clinical recommendations instead of imposing binary choices in unclear situations. A training approach consisting of two stages with random modality dropout is employed to enhance resilience in cases where one or more imaging modalities are absent. Experimental findings indicate that the suggested framework exhibits consistent performance in the presence of missing modalities, while also attaining robust diagnostic discrimination, achieving an AUC of as high as 0.98. In contrast to unimodal models, the multimodal framework generated more accurately calibrated predictions, exhibiting a significantly reduced Expected Calibration Error. Reliability-stratified analysis showed that predictions with high confidence were significantly more accurate than those with low confidence, reinforcing the clinical importance of the suggested reliability score. The proposed framework enhances practical, uncertainty-aware multimodal decision support in real clinical environments by explicitly modeling modality availability and diagnostic confidence.

Keywords: Breast cancer; Multimodal imaging; Missing-modality learning; Availability-aware fusion; Diagnostic reliability; Clinical decision support; Imaging informatics; Deep learning.