Insights into Top Paper Nominee, "OmniObject3D: Large Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation"
A Q&A with the Authors
Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin WANG, Jiawei Ren, Liang Pan, Wenyan Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, Ziwei Liu
Paper Presentation: Tuesday, 20 June, 3:20 p.m. PDT, East Exhibit Halls A-B
According to the authors of CVPR 2023 paper, “OmniObject3D: Large Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation,” their work serves as a new evaluation benchmark for numerous applications, including the potential to play a vital role in realistic 3D generation. In a Q&A interview, the research team shared insights into their work and how it advances the field of computer vision.
CVPR: Will you please share a little more about your work and results? How is it different than the standard approaches to date?
OmniObject3D is a large vocabulary 3D object dataset with massive high-quality real-scanned 3D objects. In previous approaches, most technical solutions relied heavily on unrealistic synthetic data. The appearance and distribution gaps between synthetic and real data cannot be compensated for easily, which limits their practical applications. Additionally, video or multi-view image-based datasets lack full-range 3D ground truth mesh for training and evaluation. Our work focuses on providing the community with a large-vocabulary, high-quality 3D scanned object dataset derived from the real world, which can facilitate various 3D vision tasks and downstream applications.
CVPR: How did your work outperform state-of-the-art methods? What was the key factor in these results?
Our dataset serves as a new evaluation benchmark for numerous tasks and reveals interesting observations. For example, it offers a comprehensive understanding of point cloud perception robustness under out-of-distribution challenges; it provides an extensive benchmark for novel view synthesis and surface reconstruction in both single-scene and generalizable manner; and the large vocabulary introduces challenges such as model capacity and imbalance issues to the generation task. The dataset uncovers new observations, challenges, and opportunities for future research in realistic 3D vision.
CVPR: So, what's next? What do you see as the future of your research?
Regarding the dataset itself, we are dedicated to continuously expanding and updating the dataset in order to meet a wider range of research requirements. In addition to our existing work, we plan to develop the dataset further for additional tasks, such as 2D/3D object detection and 6D pose estimation. In the era of AI-generated content (AIGC), we believe that OmniObject3D can play a vital role in contributing to realistic 3D generation.
CVPR: What more would you like to add?
3D generation is currently attracting significant attention. However, existing methods, which only utilize 2D data or crawled 3D data from the internet, result in generated objects lacking realistic geometry and texture details. The intricate geometry and authentic textures of OmniObject3D make it an ideal choice for advancing 3D generation to the next level, while complementing current training paradigms. We are actively engaged in related exploration and research efforts.
Annually, CVPR recognizes top research in the field through its prestigious “Best Paper Awards.” This year, from more than 9,000 paper submissions, the CVPR 2023 Paper Awards Committee selected 12 candidates for the coveted honor of Best Paper. Join us for the Award Session on Wednesday, 21 June at 8:30 a.m. to find out which nominees take home the distinction of “Best Paper” at CVPR 2023.