CLIP (Contrastive Language-Image Pre-Training)

Caption-Driven Explainability: Probing CNNs for Bias via CLIP
This paper introduces a novel ’network surgery’ approach that integrates CNNs with CLIP to provide caption-based explanations. By moving beyond potentially misleading saliency maps, this method identifies the dominant semantic concepts driving a model’s prediction, enabling better detection of spurious biases and improving overall model robustness.
Patrick Koller, Amil v. Dravid, Prof. Dr. Guido Schuster, Prof. Dr. Aggelos Katsaggelos
Cite Code Poster DOI IEEE Xplore arXiv
Caption-Driven Explainability: Probing CNNs for Bias via CLIP