BEIT V2: Masked Image Modeling with Vector-Quantized Visual Tokenizers

Proposes to use a semantic-rich visual tokenizer as the reconstruction target for masked prediction, providing a systematic way to promote MIM from pixel-level to semantic-level.

Can provide a more effective way of utilizing high-level semantics in self-supervised representation learning, leading to better image classification and semantic segmentation results.