ACL: Activating Capability of Linear Attention for Image Restoration

Image restoration (IR), a cornerstone of computer vision, has embarked on a new epoch with the advent of deep learning technologies. Recently, numerous CNN and Transformer-based methods have been developed, yet they frequently encounter limitations in global receptive fields and computational efficiency. To mitigate these challenges, recent studies have employed the Selective Space State Model (Mamba), which embodies both attributes. However, due to Mamba's inherent one-dimensional scanning limitations, some approaches have introduced multi-directional scanning to bolster inter-sequence correlations. Despite these enhancements, these methods still struggle with managing local pixel correlations across various directions. Moreover, the recursive computation in Mamba's SSM leads to reduced efficiency. To resolve these issues, we exploit the mathematical congruences between linear attention and SSM within the Mamba to propose a novel model based on a new design structure, ACL. This model integrates linear attention blocks instead of SSM within the Mamba, serving as the core component of encoders/decoders, and aims to preserve a global perspective while boosting computational efficiency. Furthermore, we have designed a simple yet robust local enhancement module with multi-scale dilated convolutions to extract coarse and fine features to improve local detail recovery. Experimental results confirm that our ACL model excels in classical IR tasks such as de-blurring and de-raining, while maintaining relatively low parameter counts and FLOPs.