Language Models can Solve Computer Tasks
A pre-trained large language model (LLM) agent can execute computer tasks guided by natural language using a simple prompting scheme where the agent recursively criticizes and improves its output (RCI). The RCI approach significantly outperforms existing LLM methods for automating computer tasks and surpasses supervised learning (SL) and reinforcement learning (RL) approaches on the MiniWoB++ benchmark.
Implementing RCI prompting can improve efficiency and productivity by automating repetitive tasks and assisting in complex problem-solving. This approach requires only a handful of demonstrations per task rather than tens of thousands, and without a task-specific reward function. Businesses can benefit from LLM agents in improving their workflows and processes.
SoftCLIP: Softer Cross-modal Alignment Makes CLIP Stronger
SoftCLIP is a novel approach that achieves a soft cross-modal alignment by introducing a softened target, which is generated from the fine-grained intra-modal self-similarity. The intra-modal guidance is indicative to enable two pairs have some local similarities and model many-to-many relationships between the two modalities. Extensive experiments demonstrate the effectiveness of SoftCLIP by bringing a top-1 accuracy improvement of 6.8%/7.2% over the CLIP baseline in ImageNet zero-shot classification task.
Implementing SoftCLIP can improve accuracy and reliability in vision-language pre-training on downstream tasks. The approach relaxes the strict one-to-one constraint and achieves a soft cross-modal alignment, which can be beneficial for businesses utilizing computer vision and natural language processing technologies.