Scaling Instruction-Finetuned Language Models
Explores instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on CoT data. Achieves state-of-the-art performance on several benchmarks and releases Flan-T5 checkpoints.
Instruction finetuning is a general method for improving the performance and usability of pretrained language models.
Transcending Scaling Laws with 0.1% Extra Compute
Proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. Performs on par with PaLM 540B with 2x less compute by continuing training PaLM with UL2R.
This method can help companies save on computational costs while improving the performance of existing language models.
DiffEdit: Diffusion-based semantic image editing with mask guidance
Automatically generates a mask highlighting regions of the input image that need to be edited and generates the edited image. Achieves state-of-the-art editing performance on ImageNet.
This method can help businesses improve their image editing workflows and enhance their marketing efforts by generating high-quality and relevant images for their products and services.