MMDrive: Multi-modal Remote Physiological Signal Measurement Dataset for Driver Status Monitoring

Remote physiological signal estimation, such as remote photoplethysmography (rPPG), has gained attention as a non-contact method for measuring vital signals using cameras. This technique has potential applications in telemedicine and driver monitoring systems. Several datasets have been proposed to train and evaluate models, improving the accuracy of the rPPG and heart rate estimation. However, most existing datasets have been collected in controlled laboratory environments with limited subject movements and consistent lighting conditions. Although these datasets have advanced early rPPG research, they do not consider real-world environments, and studies under unconstrained conditions remain limited. We introduce MMdrive, a multi-modal dataset designed for remote driver monitoring systems to address this gap. The dataset includes synchronized RGB, near-infrared videos of drivers operating an electric vehicle, and corresponding electrocardiogram signals. We evaluate the performance of the conventional signal processing and rPPG models using the MMDrive dataset. Specifically, our experiments include evaluations of intra- and cross-datasets and an analysis of the effectiveness of near-infrared images for remote physiological signal estimation.