Tue Apr 25 2023
Mon Apr 24 2023

HOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video

360-degree video rendering
Computer Vision
Graphics
Video rendering for gaming, virtual events, and e-commerce

HOSNeRF is a 360 free-viewpoint rendering method that reconstructs neural radiance fields for dynamic human-object-scene from a single monocular in-the-wild video. The method can render all scene details from arbitrary viewpoints, overcoming the challenge of complex object motions in human-object interactions, and humans interacting with different objects at different times. HOSNeRF outperforms SOTA approaches on two challenging datasets by a large margin of 40% ~ 50% in terms of LPIPS.

HOSNeRF can improve businesses that rely on video rendering for various applications and verticals like gaming, virtual events, and e-commerce. It can enhance the user experience by rendering high-quality images and videos from any viewpoint and help businesses showcase their products and services effectively.

Track Anything: Segment Anything Meets Videos

Interactive video object tracking and segmentation
Computer Vision
Machine Learning
Video analysis for security purposes or marketing research
Sports analytics
Autonomous vehicles

Track-Anything is a flexible and interactive tool for video object tracking and segmentation, achieving high-performance interactive tracking and segmentation in videos. Given a video sequence, with very little human participation, users can track anything they are interested in, and get satisfactory results in one-pass inference, without additional training. The tool can be useful for various applications, including video surveillance, action recognition, and autonomous vehicles.

Track-Anything can improve businesses that require video analysis for various applications, including video surveillance for security purposes or marketing research, action recognition for sports analytics, and autonomous vehicles for road safety.

Segment Anything in 3D with NeRFs

3D segmentation
Computer Vision
Graphics
Industrial design
Architecture
Virtual reality

SA3D is a novel framework that allows users to obtain the 3D segmentation result of any target object via only one-shot manual prompting in a single rendered view. The Segment Anything Model (SAM) cuts out the target object from the according view, and the obtained 2D segmentation mask is projected onto 3D mask grids via density-guided inverse rendering. 2D masks from other views are then rendered, which are mostly uncompleted but used as cross-view self-prompts to be fed into SAM again. The entire segmentation process can be completed in approximately two minutes without any engineering optimization.

SA3D can improve businesses that require 3D segmentation for various applications, including industrial design, architecture, and virtual reality. The framework can help businesses save time and resources by obtaining accurate 3D segmentation results quickly and efficiently.

AutoNeRF: Training Implicit Scene Representations with Autonomous Agents

Neural Radiance Fields (NeRF)
Machine learning
Robotics
robotics operations
environment exploration
downstream robotic tasks

AutoNeRF is a method to collect data required to train NeRFs using autonomous embodied agents, allowing an agent to explore an unseen environment efficiently and use the experience to build an implicit map representation autonomously. NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment and can be used for several downstream robotic tasks.

Businesses can use AutoNeRF to improve their robotics operations by training NeRFs to be more efficient in exploring new environments and performing tasks.

BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT

ChatGPT
reinforcement learning
Machine learning
Security
language models
security vulnerabilities
reinforcement learning fine-tuning
text manipulation

BadGPT is the first backdoor attack against RL fine-tuning in language models like ChatGPT. By injecting a backdoor into the reward model, the language model can be compromised during the fine-tuning stage, allowing an attacker to manipulate the generated text.

Businesses need to be cautious of potential security vulnerabilities in their language models, especially those that use reinforcement learning fine-tuning like ChatGPT. Regular security audits and updates to their models can help prevent these types of attacks.

Sun Apr 23 2023
Fri Apr 21 2023
Thu Apr 20 2023
Wed Apr 19 2023