HOLODIFFUSION: Training a 3D Diffusion Model using 2D Images
Diffusion models can be extended to 3D using a new diffusion setup that can be trained with 2D images for supervision and an image formation model that decouples model memory from spatial memory.
This method allows for scalable and robust training of 3D generative models, improving sample quality and fidelity to existing approaches, and can be applied to real-world data.
GPT is becoming a Turing machine: Here are some ways to program it
GPT-3 variants can perform iterative behaviours necessary to execute programs that involve loops, triggered through appropriate prompting using Regimenting Self-Attention (IRSA) in one or a combination of three ways.
IRSA has promising applications in education, allowing solving problems previously thought of as hard for LLMs, such as logical puzzles.
TaskMatrixAI: Completing Tasks by Connecting Foundation Models with Millions of APIs
TaskMatrix.AI is a new AI ecosystem that connects foundation models with millions of APIs for task completion, allowing for diversified tasks in both digital and physical domains.
This approach solves the problem of lack of domain-specific data during pre-training and errors in neural network computations on specialized tasks.
Training Language Models with Language Feedback at Scale
Pretrained language models often generate outputs that are not in line with human preferences, such as harmful text or factually incorrect summaries. This paper introduces Imitation learning from Language Feedback (ILF), a new approach that utilizes more informative language feedback. The authors show theoretically that ILF can be viewed as Bayesian Inference, similar to Reinforcement Learning from human feedback. Their experiments demonstrate that large language models accurately incorporate feedback and that finetuning with ILF scales well with the dataset size, even outperforming finetuning on human summaries.
ILF is a new approach that utilizes informative language feedback to improve pretrained language models. It shows promising results for scaling the finetuning process and achieving better performance than finetuning on human summaries alone. This could be valuable for businesses that rely on language models for natural language processing tasks, such as chatbots or automated summarization.
ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models
Large language models (LLMs) such as ChatGPT and GPT-4 have made significant progress in NLP. However, their ability to memorize, represent, and leverage commonsense knowledge has been a well-known pain point for LLMs. This paper investigates the commonsense problem in ChatGPT, showing that they can achieve good QA accuracy in commonsense tasks but struggle with certain types of knowledge. ChatGPT is knowledgeable but an inexperienced commonsense problem solver.
While large language models like ChatGPT have made significant progress in NLP, they still struggle with certain types of commonsense knowledge. This could be an area where businesses need to be aware when implementing language models to solve certain tasks. They may need to provide better commonsense guidance or other mechanisms to utilize this knowledge.