Decoupling Identity Confounders for Enhanced Facial Expression Recognition: An Information-Theoretic Approach

Facial expression recognition (FER) remains challenging due to subtle inter-class variations and significant intra-class differences, often exacerbated by identity-specific features confounding the expression features. While recent methods attempt to disentangle identity from expression, they often rely on auxiliary labels or computationally expensive image generation, limiting scalability. To address this, we propose DICE-FER (Decoupling Identity Confounders for Enhanced FER), a novel framework that decouples identity confounders from expression features through mutual information (MI) estimation without requiring labels or reconstruction. DICE-FER processes paired images with shared expressions, partitioning their features into (1) expression representations which is maximized via cross-referenced MI and (2) identity representations which is adversarially minimized for MI with expression. This dual optimization isolates identity-invariant expression cues while eliminating the need for costly generation or subject annotation. Experiments on benchmark datasets demonstrate that DICE-FER outperforms state-of-the-art methods in both disentanglement quality and recognition accuracy.