Wed Mar 08 2023
Tue Mar 07 2023
Larger language models do in-context learning differently
Natural Language Processing
Artificial Intelligence
NLP tasks
Studies how in-context learning (ICL) in LMs is affected by semantic priors versus input–label mappings.
Provides insights on how larger language models can override semantic priors and learn input-label mappings, and how instruction tuning strengthens both aspects.
Foundation Models for Decision Making: Problems, Methods, and Opportunities
Decision Making
Artificial Intelligence
Dialogue, autonomous driving, healthcare, education, and robotics
Reviews recent approaches that ground foundation models in practical decision making applications.
Provides conceptual tools and technical background for understanding the problem space and exploring new research directions.
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Data Science
Natural Language Processing
Large-scale monolingual and multilingual modeling projects
Documents the data creation and curation efforts of ROOTS corpus, a 1.6TB dataset used to train BLOOM
Provides a large multilingual dataset for training large language models, and aims to stimulate research around this large corpus.
Sun Mar 05 2023
Thu Mar 02 2023
Wed Mar 01 2023
Tue Feb 28 2023