|Table of Contents|

[1] Xing Junjie, Jia Minping, Xu Feiyun, Hu Jianzhong, et al. A method for workpiece surface small-defect detectionbased on CutMix and YOLOv3 [J]. Journal of Southeast University (English Edition), 2021, 37 (2): 128-136. [doi:10.3969/j.issn.1003-7985.2021.02.002]
Copy

A method for workpiece surface small-defect detectionbased on CutMix and YOLOv3()
基于CutMix和YOLOv3的工件表面小缺陷识别方法
Share:

Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]

Volumn:
37
Issue:
2021 2
Page:
128-136
Research Field:
Automation
Publishing date:
2021-06-20

Info

Title:
A method for workpiece surface small-defect detectionbased on CutMix and YOLOv3
基于CutMix和YOLOv3的工件表面小缺陷识别方法
Author(s):
Xing Junjie Jia Minping Xu Feiyun Hu Jianzhong
School of Mechanical Engineering, Southeast University, Nanjing 211189, China
邢俊杰 贾民平 许飞云 胡建中
东南大学机械工程学院, 南京 211189
Keywords:
machine vision image recognition deep convolutional neural network defect detection
机器视觉 图像识别 卷积神经网络 缺陷检测
PACS:
TP29
DOI:
10.3969/j.issn.1003-7985.2021.02.002
Abstract:
Surface small defects are often missed and incorrectly detected due to their small quantity and unapparent visual features. A method named CSYOLOv3, which is based on CutMix and YOLOv3, is proposed to solve such a problem. First, a four-image CutMix method is used to increase the small-defect quantity, and the process is dynamically adjusted based on the beta distribution. Then, the classic YOLOv3 is improved to detect small defects accurately. The shallow and large feature maps are split, and several of them are merged with the feature maps of the predicted branch to preserve the shallow features. The loss function of YOLOv3 is optimized and weighted to improve the attention to small defects. Finally, this method is used to detect 512 × 512 pixel images under RTX 2060Ti GPU, which can reach the speed of 14.09 frame/s, and the mAP is 71.80%, which is 5%-10% higher than that of other methods. For small defects below 64 × 64 pixels, the mAP of the method reaches 64.15%, which is 14% higher than that of YOLOv3-GIoU. The surface defects of the workpiece can be effectively detected by the proposed method, and the performance in detecting small defects is significantly improved.
针对工件表面小缺陷经常由于数量少且视觉特征不明显而导致的被漏检和错判的问题, 提出一种基于CutMix和YOLOv3的工件表面小缺陷识别方法CSYOLOv3.使用贝塔分布动态调整的CutMix方法在网络训练时动态扩充训练集中小缺陷的数量;并对YOLOv3网络进行了改进, 拆分其浅层大特征图, 取部分与预测分支的特征图融合以保留浅层的小缺陷特征;使用加权的改进损失函数对网络进行训练, 提高网络对小缺陷的重视程度和识别准确率.该方法在RTX 2060Ti GPU下对512×512像素的缺陷图片进行识别, 速度可以达到14.09帧/s, 识别mAP为71.80%, 比常用目标检测方法高出5%~10%.对于小于64×64像素的小缺陷, 方法的mAP达到64.15%, 比YOLOv3-GIoU高出14%.所提出的CSYOLOv3方法能够有效地识别工件表面缺陷, 对小缺陷的识别效果有明显提升.

References:

[1] He Y, Song K C, Meng Q G, et al. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features[J].IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493-1504. DOI:10.1109/TIM.2019.2915404.
[2] Sezgin M, Sankur B. Survey over image thresholding techniques and quantitative performance evaluation[J]. Journal of Electronic Imaging, 2004, 13(1): 146-165.
[3] Xu K, Xu Y, Zhou P, et al. Application of RNAMlet to surface defect identification of steels[J].Optics and Lasers in Engineering, 2018, 105: 110-117. DOI:10.1016/j.optlaseng.2018.01.010.
[4] You D Y, Gao X D, Katayama S. WPD-PCA-based laser welding process monitoring and defects diagnosis by using FNN and SVM[J].IEEE Transactions on Industrial Electronics, 2015, 62(1): 628-636. DOI:10.1109/TIE.2014.2319216.
[5] Liu Y, Xu K, Wang D D. Online surface defect identification of cold rolled strips based on local binary pattern and extreme learning machine[J].Metals, 2018, 8(3): 197. DOI:10.3390/met8030197.
[6] Vilar R, Zapata J, Ruiz R. An automatic system of classification of weld defects in radiographic images[J].NDT & E International, 2009, 42(5): 467-476. DOI:10.1016/j.ndteint.2009.02.004.
[7] Lecun Y, Bottou L. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
[8] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas, NV, USA, 2016: 779-788. DOI:10.1109/CVPR.2016.91.
[9] Liu W, Anguelov D, Erhan D, et al. SSD:Single shot MultiBox detector[M]//Computer Vision—ECCV 2016. Cham: Springer International Publishing, 2016: 21-37. DOI:10.1007/978-3-319-46448-0_2.
[10] Girshick R, Donahue J, Darrell T, et al. Region-based convolutional networks for accurate object detection and segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(1): 142-158. DOI:10.1109/TPAMI.2015.2437384.
[11] Li J Y, Su Z F, Geng J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network[J].IFAC-Papers OnLine, 2018, 51(21): 76-81. DOI:10.1016/j.ifacol.2018.09.412.
[12] Wei R F, Bi Y B. Research on recognition technology of aluminum profile surface defects based on deep learning[J]. Materials, 2019, 12(10): 1681.
[13] Li Y T, Huang H S, Xie Q S, et al. Research on a surface defect detection algorithm based on MobileNet-SSD[J].Applied Sciences, 2018, 8(9): 1678. DOI:10.3390/app8091678.
[14] Xue Y D, Li Y C. A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects[J].Computer-Aided Civil and Infrastructure Engineering, 2018, 33(8): 638-654. DOI:10.1111/mice.12367.
[15] Du W Z, Shen H Y, Fu J Z, et al. Approaches for improvement of the X-ray image defect detection of automobile casting aluminum parts based on deep learning[J].NDT & E International, 2019, 107: 102144. DOI:10.1016/j.ndteint.2019.102144.
[16] Yun S, Han D, Chun S, et al. CutMix: Regularization strategy to train strong classifiers with localizable features[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Seoul, South Korea, 2019: 6022-6031. DOI:10.1109/ICCV.2019.00612.
[17] Redmon J, Farhadi A. YOLOv3:An incremental improvement[EB/OL].(2018)[2020-10-20]. https: //arxiv.org/abs/1804.02767
[18] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas, NV, USA, 2016: 770-778. DOI:10.1109/CVPR.2016.90.
[19] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning. Lille, France, 2015: 448-456.
[20] Rezatofighi H, Tsoi N, Gwak J, et al. Generalized intersection over union: A metric and a loss for bounding box regression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, CA, USA, 2019: 658-666. DOI:10.1109/CVPR.2019.00075.
[21] Han Z D. Dyna: A method of momentum for stochastic optimization[EB/OL].(2018)[2020-10-20]. https: //arxiv.org/abs/1805.04933.
[22] Arthur D, Vassilvitskii S. K-means++: The advantages of careful seeding[C]//Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. Philadelphia, USA, 2007: 1027-1035.
[23] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI:10.1109/TPAMI.2016.2577031.
[24] Everingham M, van Gool L, Williams C K I, et al. The pascal visual object classes(voc)challenge[J]. International Journal of Computer Vision, 2010, 88(2): 303-338.
[25] Everingham M, Eslami S M A, van Gool L, et al. The pascal visual object classes challenge: A retrospective[J]. International Journal of Computer Vision, 2015, 111(1): 98-136. DOI:10.1007/s11263-014-0733-5.

Memo

Memo:
Biographies: Xing Junjie(1997—), male, graduate; Jia Minping(corresponding author), male, doctor, professor, mpjia@seu.edu.cn.
Foundation item: The National Natural Science Foundation of China(No. 52075095).
Citation: Xing Junjie, Jia Minping, Xu Feiyun, et al. A method for workpiece surface small-defect detection based on CutMix and YOLOv3[J].Journal of Southeast University(English Edition), 2021, 37(1):128-136.DOI:10.3969/j.issn.1003-7985.2021.02.002.
Last Update: 2021-06-20