MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining

This paper presents a simple yet effective framework MaskCLIP, which incorporates a newly proposed masked self-distillation into contrastive language-image pretraining. It achieves superior results in linear probing, finetuning as well as the zero-shot performance.

Businesses that deal with language and image data can use MaskCLIP to improve their pretraining performance and achieve better downstream task results.