Visual Recognition by Request

Chufeng Tang · Lingxi Xie · Xiaopeng Zhang · Xiaolin Hu · Qi Tian

West Building Exhibit Halls ABC 277
[ Abstract ]
Wed 21 Jun 4:30 p.m. PDT — 6 p.m. PDT


Humans have the ability of recognizing visual semantics in an unlimited granularity, but existing visual recognition algorithms cannot achieve this goal. In this paper, we establish a new paradigm named visual recognition by request (ViRReq) to bridge the gap. The key lies in decomposing visual recognition into atomic tasks named requests and leveraging a knowledge base, a hierarchical and text-based dictionary, to assist task definition. ViRReq allows for (i) learning complicated whole-part hierarchies from highly incomplete annotations and (ii) inserting new concepts with minimal efforts. We also establish a solid baseline by integrating language-driven recognition into recent semantic and instance segmentation methods, and demonstrate its flexible recognition ability on CPP and ADE20K, two datasets with hierarchical whole-part annotations.

Chat is not available.