GPTScore: Evaluate as You Desire
Proposes a novel evaluation framework, GPTSCORE, which utilizes the emergent abilities (e.g., zero-shot instruction) from generative pre-trained models to score generated texts.
Provides a new evaluation framework for generated texts that utilizes pre-trained models to achieve customized, multi-faceted evaluation without the need for annotated samples. The approach can effectively allow businesses to assess the quality of generated text.
Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions
Develops a practical, memory-efficient algorithm which approximately recovers Shampoo performance in three modern DL settings (ImageNet, Librispeech, etc).
Introduces a low-rank sketching approach to adaptive regularization methods in deep learning tasks, reducing memory and compute requirements of maintaining a matrix preconditioner. The method performs well in large scale benchmarks such as ImageNet and Librispeech.