[1] MENG Jiahao, LIU Weirong, SHI Changhong, LI Zhijun, et al. An image inpainting method based on multiple receptive fields and dynamic matching of damaged patterns [J]. Journal of Southeast University (English Edition), 2026, 42 (1): 121-130. [doi:10.3969/j.issn.1003-7985.2026.01.012]
Copy
An image inpainting method based on multiple receptive fields and dynamic matching of damaged patterns(
)
基于多感受野与破损模式动态匹配的图像修复方法
)Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]
- Volumn:
- 42
- Issue:
- 2026 1
- Page:
- 121-130
- Research Field:
- Computer Science and Engineering
- Publishing date:
- 2026-03-20
Info
- Title:
- An image inpainting method based on multiple receptive fields and dynamic matching of damaged patterns
- 基于多感受野与破损模式动态匹配的图像修复方法
- Author(s):
- MENG Jiahao, LIU Weirong, SHI Changhong, LI Zhijun, LIU Jie
- School of Automation and Electrical Engineering, Lanzhou University of Technology, Lanzhou 730050, China
- Keywords:
- image inpainting; generative adversarial networks; multiple receptive fields (MRFs); dynamic matching of damaged patterns; decoder with fast Fourier convolutional
- 图像修复; 生成对抗网络; 多感受野; 破损模式动态匹配; 快速傅里叶卷积解码器
- PACS:
- TP391.41
- DOI:
- 10.3969/j.issn.1003-7985.2026.01.012
- Abstract:
- Current image inpainting models are primarily designed to achieve a large receptive field (RF) using refinement networks to incorporate different scales. However, these models fail to adapt the use of different RFs to the specific patterns of image damage, resulting in artifacts and semantic information confusion in repaired images. To address the problems of artifacts and semantic information confusion, inspired by different sensitivities of different RFs to inpainting the same image damaged patterns, this study proposes an image inpainting method based on multiple receptive fields (MRFs) and dynamic matching of damaged patterns. First, the parallel filter banks are used to extract the MRF feature groups. Second, the features are dynamically weighted and screened, guided by the mask image, to construct a relationship that adaptively matches the most relevant RF to each specific damaged pattern. A fast Fourier convolution based decoder is used to enhance the fusion of global contextual features during the reconstruction of high dimensional features into low dimensional images. Comparative experimental results show that the proposed method achieves better subjective and objective inpainting results on three public datasets: Paris StreetView, CelebA-HQ, and Places2.
- 当前图像修复模型设计时多以大感受野为基础,通过细修复网络实现不同感受野的利用,忽略了不同感受野与图像破损模式间的适配性,进而造成修复图像出现伪影、语义信息混乱等问题。针对伪影和语义信息混乱问题,受不同感受野对同一图像破损模式修复敏感性不同的启发,本文提出了一种基于多感受野与破损模式动态匹配的图像修复方法。首先,该方法通过并联的滤波器组实现统一尺度多感受野特征组的提取。其次,以掩码图像为约束条件,通过对多感受野特征组的动态加权和筛选,构建起感受野与破损模式间的动态匹配关系。最后,设计了基于快速傅里叶卷积的解码器,用以增强解码过程中高维特征向低维图像恢复时全局上下文特征的融合能力。对比实验结果表明,该方法在Paris StreetView、CelebA-HQ和Places2三个公共数据集上可以获得更好的主客观修复结果。
References:
[1]WANG H, WANG X, LIAO R X, et al. Ship identification and tracking for navigable bridge based on improved YOLOv8 and ByteTrack[J]. Journal of Southeast University (Natural Science Edition), 2025, 55(5): 1380-1387. (in Chinese)
[2]JIA J W, NI Y H, MAO J X, et al. Geometric parameter identification of bridge precast box girder sections based on deep learning and computer vision[J]. Journal of Southeast University (English Edition), 2025, 41(3): 278-285.
[3]XIANG H Y, ZOU Q, NAWAZ M ALI, et al. Deep learning for image inpainting: A survey[J]. Pattern Recognition, 2023, 134: 109046.
[4]HUANG J B, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4): 1-10.
[5]ZHAO X, WANG L M, ZHANG Y F, et al. A review of convolutional neural networks in computer vision[J]. Artificial Intelligence Review, 2024, 57(4): 99.
[6]HUO G Y, SHENG X L, SHI S X. Unsupervised underwater image enhancement with global background light intensity estimation and contrast adjustment[J]. Journal of Southeast University (Natural Science Edition), 2025, 55(1): 297-305. (in Chinese)
[7]LIU J L, GONG M G, TANG Z D, et al. Deep image inpainting with enhanced normalization and contextual attention[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(10): 6599-6614.
[8]NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: Structure guided image inpainting using edge prediction[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). Seoul, Korea, 2019: 3265-3274.
[9]ZHANG R S, QUAN W Z, ZHANG Y, et al. W-net: Structure and texture interaction for image inpainting[J]. IEEE Transactions on Multimedia, 2022, 25: 7299-7310.
[10]LI Y, ZHAI J, LU W, et al. Image inpainting with aggregated convolution progressive network[J]. IET Image Processing, 2025, 19(1): e13318.
[11]LIU S, CHEN J Y, DING X L, et al. Progressive Reverse Attention Network for image inpainting detection and localization[J]. Computer Vision and Image Understanding, 2025, 259: 104407.
[12]ZHENG C X, CHAM T J, CAI J F, et al. Bridging global context interactions for high-fidelity image completion[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA, 2022: 11502-11512.
[13]YU J H, LIN Z, YANG J M, et al. Generative image inpainting with contextual attention[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, 2018: 5505-5514.
[14]LIU H Y, JIANG B, SONG Y B, et al. Rethinking image inpainting via a mutual encoder-decoder with feature equalizations[M]//Computer Vision-ECCV 2020. Cham, Switzerland: Springer International Publishing, 2020: 725-741.
[15]LIU G L, REDA F A, SHIH K J, et al. Image inpainting for irregular holes using partial convolutions[C]//Computer Vision-ECCV 2018. Cham, Switzerland: Springer, 2018: 89-105.
[16]JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Computer Vision-ECCV 2016. Cham, Switzerland: Springer, 2016: 694-711.
[17]GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, 2016: 2414-2423.
[18]DOERSCH C, SINGH S, GUPTA A, et al. What makes paris look like paris？[J]. Communications of the ACM, 2015, 58(12): 103-110.
[19]LIU Z W, LUO P, WANG X G, et al. Deep learning face attributes in the wild[C]//2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile, 2015: 3730-3738.
[20]ZHOU B L, LAPEDRIZA A, KHOSLA A, et al. Places: A 10 million image database for scene recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6): 1452-1464.
[21]QUAN W Z, ZHANG R S, ZHANG Y, et al. Image inpainting with local and global refinement[J]. IEEE Transactions on Image Processing, 2022, 31: 2405-2420.
[22]WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[23]HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium[C]//Proceedings of the International Conference on Neural Information Processing Systems. Long Beach, CA, USA, 2017: 6629-6640.
[24]ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, 2018: 586-595.
[25]ZHU M Y, HE D L, LI X, et al. Image inpainting by end- to-end cascaded refinement with mask awareness[J]. IEEE Transactions on Image Processing, 2021, 30: 4855-4866.
[26]ZENG Y H, FU J L, CHAO H Y, et al. Aggregated contextual transformations for high-resolution image inpainting[J]. IEEE Transactions on Visualization and Computer Graphics, 2023, 29(7): 3266-3280.
[27]JAIN J, ZHOU Y Q, YU N, et al. Keys to better image inpainting: Structure and texture go hand in hand[C]//2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI, USA, 2023: 208-217.
[28]ZUO Z W, ZHAO L, LI A L, et al. Generative image inpainting with segmentation confusion adversarial training and contrastive learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(3): 3888-3896.
Memo
- Memo:
-
Received: 2025-07-03; Revised: 2025-09-21.
Biographies: MENG Jiahao(1993—), male, Ph.D.candidate; LIU Weirong(corresponding author), male, doctor, professor, liuwr@lut.edu.cn.
Foundation items: The National Natural Science Foundation of China(No.62261032), the Central Government Guiding Funds for Local Science and Technology Development Program (No.25ZYJA026).
Citation: MENG Jiahao, LIU Weirong, SHI Changhong, et al. An image inpainting method based on multiple receptive fields and dynamic matching of damaged patterns[J]. Journal of Southeast University (English Edition), 2026, 42(1): 121-130. DOI: 10. 3969/j. issn. 1003-7985. 2026. 01. 012.