ASPS: Augmented Segment Anything Model for Polyp Segmentation

摘要

Segment Anything Model (SAM) offers strong generic segmentation capability, but its transfer to endoscopic imagery is limited by the domain gap from natural images. Its Transformer encoder emphasizes global, low-frequency information and may miss local polyp details, while its out-of-distribution predictions can be accompanied by poorly calibrated confidence. ASPS addresses these limitations without relying on prompts. Cross-branch Feature Augmentation (CFA) adds a trainable CNN encoder alongside the frozen ViT encoder and integrates local, high-frequency, multi-scale features through cross-branch attention. Uncertainty-guided Prediction Regularization (UPR) adapts selected normalization parameters and uses SAM's IoU score together with pixel-level uncertainty to guide confidence-aware training. The two modules jointly enhance feature representation and domain generalization for automatic polyp segmentation.

主要结果

Experiments on CVC-ClinicDB, Kvasir-SEG, CVC-ColonDB, ETIS, and EndoScene show that ASPS consistently improves SAM-based baselines. The ViT-H configuration achieved average Dice and IoU scores of 0.890 and 0.817, compared with 0.870 and 0.804 for Polyp-PVT. It obtained Dice scores of 0.951 on CVC-ClinicDB, 0.920 on Kvasir-SEG, 0.799 on CVC-ColonDB, 0.861 on ETIS, and 0.919 on EndoScene. Ablation experiments confirm complementary contributions from the CNN branch, cross-branch attention, multi-level feature fusion, and uncertainty-guided regularization.

@inproceedings{li2024asps,
  author    = {Li, Huiqian and Zhang, Dingwen and Yao, Jieru and Han, Longfei and Li, Zhongyu and Han, Junwei},
  title     = {{ASPS}: Augmented Segment Anything Model for Polyp Segmentation},
  booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
  series    = {Lecture Notes in Computer Science},
  volume    = {15009},
  pages     = {118--128},
  year      = {2024},
  doi       = {10.1007/978-3-031-72114-4_12}
}