Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner

Proposes Prompt-based Continued Pre-training, which consistently improves the performance of prompt-based fine-tuning (up to 20.1% absolute).

This approach can improve the performance of prompt-based fine-tuning with up to 20.1% absolute improvement. It can also simplify the process, eliminating the need for iterative processes and extra data augmentation.