Abstract & Article Details
Research Article • Vol.7, Issue 8 • ISSN: 2766-2276 • Open Access • CC BY 4.0
Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub
Abstract
Background: Automated lesion classification and interpretation of pediatric abdominal X-rays is clinically important but constrained by data scarcity and strict privacy regulations. Synthetic image-caption pair datasets offer a practical solution. Methods: We used the AI Hub synthetic pediatric abdominal X-ray dataset (10,000 image-caption pairs, five disease classes, stratified 8:1:1 split with atomic pair splitting to prevent data leakage) to develop a BioMedCLIP-based multimodal system. The system integrates a ViT-B/16 visual encoder fine-tuned with InfoNCE contrastive loss, an MLP classification head with MC Dropout for uncertainty quantification (T = 20 forward passes), and a three-tier confidence-conditioned report generation module. Clinical performance was assessed using accuracy, sensitivity, specificity, precision, and F1-score; report generation was assessed with BLEU-4, ROUGE-L, and CIDEr. ResNet-50 (B1) and DenseNet-121 (B2) served as classification baselines; template-based (B_tmpl) and LLaVA-Med (B3) as generation baselines. Results: The proposed model achieved accuracy 97.95%, sensitivity 91.57%, specificity 96.28%, and macro F1-score 93.49%, outperforming B1 (+5.04 pp) and B2 (+2.55 pp) in accuracy. For report generation, BLEU-4 6.46, ROUGE-L 20.51, and CIDEr 49.26 exceeded both baselines on all metrics. Grad-CAM analysis confirmed anatomically plausible attention patterns. Conclusions: Multimodal vision-language contrastive learning on synthetic image-caption data yields consistent gains over single-modality baselines. The sim-to-real distribution gap and the need for external clinical validation are identified as the primary barriers to clinical translation.
Keywords
Research Topics
How to Cite
Article Information
| Journal | Journal of Biomedical Research & Environmental Sciences (JBRES) |
|---|---|
| ISSN | 2766-2276 |
| DOI | DOI 10.37871/jbres2321 |
| Volume / Issue | Vol. 7, Issue 8 |
| Received | August 16, 2026 |
| Accepted | August 1, 2026 |
| Published | August 5, 2026 |
| Article Type | Research Article |
| Pages | 1-13 |
| License | CC BY 4.0 — Open Access |
| Publisher | SciRes Literature LLC, Sheridan, WY, USA |
| Language | English |
Published under CC BY 4.0 — free to share, copy, adapt, and redistribute with attribution.