作者：Haijin Zeng Kai Feng Shaoguang Huang Jiezhang Cao Yongyong Chen Hongyan Zhang Hiep Luong Wilfried Philips
Hyperspectral imaging systems that use multispectral filter arrays (MSFA)capture only one spectral component in each pixel. Hyperspectral demosaicing isused to recover the non-measured components. While deep learning methods haveshown promise in this area, they still suffer from several challenges,including limited modeling of non-local dependencies, lack of consideration ofthe periodic MSFA pattern that could be linked to periodic artifacts, anddifficulty in recovering high-frequency details. To address these challenges,this paper proposes a novel de-mosaicing framework, the MSFA-frequency-awareTransformer network (FDM-Net). FDM-Net integrates a novel MSFA-frequency-awaremulti-head self-attention mechanism (MaFormer) and a filter-based Fourierzero-padding method to reconstruct high pass components with greater difficultyand low pass components with relative ease, separately. The advantage ofMaformer is that it can leverage the MSFA information and non-localdependencies present in the data. Additionally, we introduce a joint spatialand frequency loss to transfer MSFA information and enhance training onfrequency components that are hard to recover. Our experimental resultsdemonstrate that FDM-Net outperforms state-of-the-art methods with 6dB PSNR,and reconstructs high-fidelity details successfully.