DO FUSION MODULES AND COMPOUND LOSSES REALLY HELP? A MULTI-SEED ABLATION OF A HYBRID CNN–TRANSFORMER MODEL FOR 3D BRAIN TUMOR SEGMENTATION
- Authors
-
-
G. X. Abdukadirova
Fergana State Technical University -
M. I. Abduqodirova
Fergana State Technical University
-
- Keywords:
- brain tumor segmentation hybrid CNN–Transformer feature fusion cross-attention compound loss ablation study multi-seed evaluation reproducibility negative results BraTS 2021
- Abstract
- Hybrid CNN–Transformer models for brain tumor segmentation are usually justified by ablation studies in which each variant is trained once, although run-to-run variation can be as large as the reported gains. This study re-examines the components of the HyCTSeg model - a parallel CNN and Swin Transformer encoder with bidirectional cross-attention fusion (CAFF) and a four-component loss (Dice, cross-entropy, focal, Tversky) - with a statistically powered design. Nine variants were each trained with four random seeds on BraTS 2021 (32 independent runs, 150 epochs) on an identical data split, and compared by paired t-tests with Benjamini–Hochberg correction. The multi-seed design reduced the standard deviation to 0.0006–0.0018 Dice. The full model ranked third of five architectures (0.8593 ± 0.0015); the Swin-only encoder was best (0.8645 ± 0.0016) and exceeded the full model by 0.0052 (p = 0.0096), and simple additive fusion exceeded it by 0.0032 (p = 0.0107), both significant after correction. The differences were concentrated in the enhancing tumor region. The compound loss did not outperform Dice loss alone (−0.0004, p = 0.38). These negative results show that single-run ablations can overstate the value of architectural components and argue for multi-seed evaluation as standard practice.
- References
-
1. Menze B. H., Jakab A., Bauer S. et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) // IEEE Transactions on Medical Imaging. – 2015. – Vol. 34, No. 10. – P. 1993–2024.
2. Baid U., Ghodasara S., Mohan S. et al. The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification // arXiv:2107.02314. – 2021.
3. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation // MICCAI 2015. – LNCS 9351. – P. 234–241.
4. Çiçek Ö., Abdulkadir A., Lienkamp S. S. et al. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation // MICCAI 2016. – LNCS 9901. – P. 424–432.
5. Isensee F., Jaeger P. F., Kohl S. A. A. et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation // Nature Methods. – 2021. – Vol. 18. – P. 203–211.
6. Wang W., Chen C., Ding M. et al. TransBTS: Multimodal Brain Tumor Segmentation Using Transformer // MICCAI 2021. – LNCS 12901. – P. 109–119.
7. Hatamizadeh A., Nath V., Tang Y. et al. Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images // BrainLes 2021. – LNCS 12962. – 2022. – P. 272–284.
8. Zhang Y., Liu H., Hu Q. TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation // MICCAI 2021. – LNCS 12901. – P. 14–24.
9. Bouthillier X., Delaunay P., Bronzi M. et al. Accounting for Variance in Machine Learning Benchmarks // Proceedings of MLSys. – 2021. – Vol. 3. – P. 747–769.
10. Maier-Hein L., Eisenmann M., Reinke A. et al. Why rankings of biomedical image analysis competitions should be interpreted with care // Nature Communications. – 2018. – Vol. 9. – Article 5217.
11. Lipton Z. C., Steinhardt J. Troubling Trends in Machine Learning Scholarship // ACM Queue. – 2019. – Vol. 17, No. 1. – P. 45–77.
12. Isensee F., Wald T., Ulrich C. et al. nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation // MICCAI 2024. – LNCS 15009. – P. 488–498.
13. Milletari F., Navab N., Ahmadi S.-A. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation // 3DV. – 2016. – P. 565–571.
14. Lin T.-Y., Goyal P., Girshick R. et al. Focal Loss for Dense Object Detection // IEEE ICCV. – 2017. – P. 2980–2988.
15. Salehi S. S. M., Erdogmus D., Gholipour A. Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks // MLMI 2017. – LNCS 10541. – P. 379–387.
16. Ma J., Chen J., Ng M. et al. Loss odyssey in medical image segmentation // Medical Image Analysis. – 2021. – Vol. 71. – Article 102035.
17. Benjamini Y., Hochberg Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing // Journal of the Royal Statistical Society: Series B. – 1995. – Vol. 57, No. 1. – P. 289–300.