What Happened 3 Seconds Ago? Inferring the Past With Thermal Imaging

Zitian Tang · Wenjie Ye · Wei-Chiu Ma · Hang Zhao

West Building Exhibit Halls ABC 060
[ Abstract ]
[ Paper PDF [ Slides [ Poster
Thu 22 Jun 10:30 a.m. PDT — noon PDT


Inferring past human motion from RGB images is challenging due to the inherent uncertainty of the prediction problem. Thermal images, on the other hand, encode traces of past human-object interactions left in the environment via thermal radiation measurement. Based on this observation, we collect the first RGB-Thermal dataset for human motion analysis, dubbed Thermal-IM. Then we develop a three-stage neural network model for accurate past human pose estimation. Comprehensive experiments show that thermal cues significantly reduce the ambiguities of this task, and the proposed model achieves remarkable performance. The dataset is available at

Chat is not available.