Nonparametric Masked Language Modeling

Introduces NPM, the first nonparametric MLM that replaces its softmax with a nonparametric distribution over every phrase in a reference corpus.

Can improve language models' ability to predict rare tokens or phrases, especially non-Latin script, and outperforms larger parametric models.