Amlpf-clip: Adaptive Prompting and Distilled Learning for Imbalanced Histopathological Image Classification

TY - JOUR

T1 - Amlpf-clip

T2 - Adaptive Prompting and Distilled Learning for Imbalanced Histopathological Image Classification

AU - Yao, Xizhang

AU - Yue, Guanghui

AU - Deng, Jeremiah D.

AU - Lin, Hanhe

AU - Zhou, Wei

N1 - Publisher Copyright: © 2013 IEEE.

PY - 2025/10/9

Y1 - 2025/10/9

N2 - Histopathological image classification (HIC) plays a pivotal role in computer-aided diagnosis, enabling lesion characterization (e.g., tumor grading) and survival outcome prediction. Despite recent advances in HIC, existing methods still face challenges in integrating domain-specific knowledge, addressing class imbalance, and ensuring computational efficiency. To address these challenges, we propose AMLPF-CLIP, an enhanced CLIP-based framework for HIC featuring three key innovations. First, we introduce an Adaptive Multi-Level Prompt Fusion (AMLPF) strategy that leverages three levels of textual prompts: class labels, basic descriptions, and GPT-4o-generated detailed pathological features for enhanced semantic representation and cross-modal alignment. Second, we design a class-balanced resampling method that dynamically adjusts sampling weights based on both data imbalance and classification performance, targeting underrepresented, low-confidence classes. Third, we develop a Knowledge Distillation (KD) technique that leverages output-level alignment via L2 loss, transferring knowledge from a large Vision Transformer (ViT-L/16) to a lightweight ResNet-50-based CLIP model. Extensive experiments on three public datasets demonstrate that AMLPF-CLIP consistently outperforms eleven state-of-the-art methods, achieving accuracy improvements of 1.19% on Chaoyang, 2.64% on BreaKHis, and 0.90% on LungHist700. AMLFP-CLIP also demonstrates improved robustness and efficiency, highlighting its practical applicability.

AB - Histopathological image classification (HIC) plays a pivotal role in computer-aided diagnosis, enabling lesion characterization (e.g., tumor grading) and survival outcome prediction. Despite recent advances in HIC, existing methods still face challenges in integrating domain-specific knowledge, addressing class imbalance, and ensuring computational efficiency. To address these challenges, we propose AMLPF-CLIP, an enhanced CLIP-based framework for HIC featuring three key innovations. First, we introduce an Adaptive Multi-Level Prompt Fusion (AMLPF) strategy that leverages three levels of textual prompts: class labels, basic descriptions, and GPT-4o-generated detailed pathological features for enhanced semantic representation and cross-modal alignment. Second, we design a class-balanced resampling method that dynamically adjusts sampling weights based on both data imbalance and classification performance, targeting underrepresented, low-confidence classes. Third, we develop a Knowledge Distillation (KD) technique that leverages output-level alignment via L2 loss, transferring knowledge from a large Vision Transformer (ViT-L/16) to a lightweight ResNet-50-based CLIP model. Extensive experiments on three public datasets demonstrate that AMLPF-CLIP consistently outperforms eleven state-of-the-art methods, achieving accuracy improvements of 1.19% on Chaoyang, 2.64% on BreaKHis, and 0.90% on LungHist700. AMLFP-CLIP also demonstrates improved robustness and efficiency, highlighting its practical applicability.

KW - CLIP

KW - Histopathological image classification

KW - Imbalanced classification

KW - Knowledge distillation

KW - Multimodal learning

UR - https://www.scopus.com/pages/publications/105019556658

U2 - 10.1109/JBHI.2025.3619343

DO - 10.1109/JBHI.2025.3619343

M3 - Article

C2 - 41066279

SN - 2168-2208

SP - 1

EP - 12

JO - IEEE Journal of Biomedical and Health Informatics

JF - IEEE Journal of Biomedical and Health Informatics

ER -