Large Scale Language Modeling with Recurrent Neural Networks
This paper explores recent advances in Recurrent Neural Networks for large scale Language Modeling and extends current models to deal with two key challenges present in this task.
The paper provides insights on techniques such as character Convolutional Neural Networks or Long-Short Term Memory and offers models for the NLP and ML community to study and improve upon.
Big Model Training
This paper explores big model training and dives into training objectives and methodologies. The existing training methodologies are summarized into three main categories: training parallelism, memory-saving technologies, and model sparsity design.
The paper provides insights on how to leverage web-scale data to develop incredibly large models based on self-supervised learning and how to make big model training a reality. It also provides a continuously updated paper list of big model training.
Building a Social, Informative Open-Domain Dialogue Agent
This paper presents Chirpy Cardinal, an open-domain social chatbot that aims to be both informative and conversational. The chatbot integrates controlled neural generation with scaffolded, hand-written dialogue to let both the user and bot take turns driving the conversation.
The paper provides insights on building a socially fluent chatbot and details Chirpy Cardinal's performance in the Alexa Prize Socialbot Grand Challenge.