Event Quality Score (EQS): Assessing the Realism of Simulated Event Camera Streams via Distance in Latent Space

Event cameras promise a paradigm shift in vision sensing with their low latency, high dynamic range, and asynchronous nature of events. Unfortunately, the scarcity of high-quality labeled datasets hinders their widespread adoption in deep-learning-driven computer vision. To mitigate this, several simulators have been proposed to generate synthetic event data for training models for detection and estimation tasks. However, the fundamentally different sensor design of event cameras compared to traditional frame-based cameras poses a challenge for accurate simulation. As a result, most simulated data fails to mimic data captured by real event cameras. Inspired by existing work on using deep features for image comparison, we introduce Event Quality Score (EQS), a quality metric that utilizes activations of the RVT architecture. Through sim-to-real experiments on the DSEC driving dataset, it is shown that a higher EQS score implies improved generalization to real-world data after training on simulated events. Thus optimizing for EQS can lead to the development of more realistic event camera simulators, effectively reducing the simulation gap.