[1] Zhang Qinglei, Yang Zhiwei, Duan Jianguo, Liu Zhen, et al. Three-dimensional visualization interactive systemfor digital twin workshop [J]. Journal of Southeast University (English Edition), 2021, 37 (2): 137-152. [doi:10.3969/j.issn.1003-7985.2021.02.003]

Copy

Three-dimensional visualization interactive systemfor digital twin workshop()

Share：

Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]

Volumn:

37

Issue:

2021 2

Page:

137-152

Research Field:

Mechanical Engineering

Publishing date:

2021-06-20

Info

Title:

Three-dimensional visualization interactive systemfor digital twin workshop

Author(s):

Zhang Qinglei;  Yang Zhiwei;  Duan Jianguo;  Liu Zhen;  Qin Jiyun

School of Logistics Engineering, Shanghai Maritime University, Shanghai 201306, China

Keywords:

digital twin workshop;  three-dimensional visualization;  human-computer interaction;  data driven;  OPC UA(object linking and embedding for process control unified architecture)

PACS:

TH166;TP399

DOI:

10.3969/j.issn.1003-7985.2021.02.003

Abstract:

To improve the human-physical-virtual coordination and integration of the digital twin workshop, 3D visual monitoring and human-computer interaction of the digital twin workshop was studied. First, a novel 6D model of the 3D visualization interactive system for digital twin workshops is proposed. As the traditional 5D digital twin model ignores the importance of human-computer interaction, a new dimension of the user terminal was added. A hierarchical real-time data-driven mapping model for the workshop production process is then proposed. Moreover, a real-time data acquisition method for the industrial Internet of things is proposed based on OPC UA(object linking and embedding for process control unified architecture). Based on the 6D model of the system, the process of creating a 3D visualization virtual environment based on virtual reality is introduced, in addition to a data-driven process based on the data management cloud platform. Finally, the 6D model of the system was confirmed using the blade rotor test workshop as the object, and a 3D visualization interactive system is developed. The results show that the system is more transparent, real-time, data-driven and more efficient, as well as promotes the coordination and integration of human-physical-virtual, which has practical significance for developing digital twin workshops.

References:

[1] Tao F, Qi Q L, Liu A, et al. Data-driven smart manufacturing[J]. Journal of Manufacturing Systems, 2018, 48: 157-169. DOI:10.1016/j.jmsy.2018.01.006.
[2] Jones D, Snider C, Nassehi A, et al. Characterising the digital twin: A systematic literature review[J].CIRP Journal of Manufacturing Science and Technology, 2020, 29: 36-52. DOI:10.1016/j.cirpj.2020.02.002.
[3] Lu Y Q, Liu C, Wang K I K, et al. Digital twin-driven smart manufacturing: Connotation, reference model, applications and research issues[J].Robotics and Computer-Integrated Manufacturing, 2020, 61: 101837. DOI:10.1016/j.rcim.2019.101837.
[4] Mourtzis D, Vlachou E, Milas N. Industrial big data as a result of IoT adoption in manufacturing[J]. Procedia CIRP, 2016, 55: 290-295. DOI:10.1016/j.procir.2016.07.038.
[5] Hashem I A T, Yaqoob I, Anuar N B, et al. The rise of “big data” on cloud computing: Review and open research issues[J]. Information Systems, 2015, 47: 98-115. DOI:10.1016/j.is.2014.07.006.
[6] Pivoto D G S, de Almeida L F F, da Rosa Righi R, et al. Cyber-physical systems architectures for industrial Internet of things applications in Industry 4.0: A literature review[J]. Journal of Manufacturing Systems, 2021, 58: 176-192. DOI:10.1016/j.jmsy.2020.11.017.
[7] Neto A A, Deschamps F, da Silva E R, et al. Digital twins in manufacturing: An assessment of drivers, enablers and barriers to implementation[J]. Procedia CIRP, 2020, 93: 210-215. DOI:10.1016/j.procir.2020.04.131.
[8] Schleich B, Anwer N, Mathieu L, et al. Shaping the digital twin for design and production engineering[J]. CIRP Annals, 2017, 66(1): 141-144. DOI:10.1016/j.cirp.2017.04.040.
[9] Lim K Y H, Zheng P, Chen C H, et al. A digital twin-enhanced system for engineering product family design and optimization[J].Journal of Manufacturing Systems, 2020, 57: 82-93. DOI:10.1016/j.jmsy.2020.08.011.
[10] Meraghni S, Terrissa L S, Yue M L, et al. A data-driven digital-twin prognostics method for proton exchange membrane fuel cell remaining useful life prediction[J]. International Journal of Hydrogen Energy, 2021, 46(2): 2555-2564. DOI:10.1016/j.ijhydene.2020.10.108.
[11] Aivaliotis P, Georgoulias K, Arkouli Z, et al. Methodology for enabling digital twin using advanced physics-based modelling in predictive maintenance[J]. Procedia CIRP, 2019, 81: 417-422. DOI:10.1016/j.procir.2019.03.072.
[12] Schützer K, de Bertazzi J D A, Sallati C, et al. Contribution to the development of a digital twin based on product lifecycle to support the manufacturing process[J]. Procedia CIRP, 2019, 84: 82-87. DOI:10.1016/j.procir.2019.03.212.
[13] Tao F, Zhang M. Digital twin shop-floor: A new shop-floor paradigm towards smart manufacturing[J].IEEE Access, 2017, 5: 20418-20427. DOI:10.1109/ACCESS.2017.2756069.
[14] Qi Q L, Tao F, Hu T L, et al. Enabling technologies and tools for digital twin[J]. Journal of Manufacturing Systems, 2021, 58: 3-21. DOI:10.1016/j.jmsy.2019.10.001.
[15] Zhuang C B, Miao T, Liu J H, et al. The connotation of digital twin, and the construction and application method of shop-floor digital twin[J]. Robotics and Computer-Integrated Manufacturing, 2021, 68: 102075. DOI:10.1016/j.rcim.2020.102075.
[16] Liu Q, Leng J W, Yan D X, et al. Digital twin-based designing of the configuration, motion, control, and optimization model of a flow-type smart manufacturing system[J]. Journal of Manufacturing Systems, 2021, 58: 52-64. DOI:10.1016/j.jmsy.2020.04.012.
[17] de D’Amico D, Ekoyuncu J, Addepalli S, et al. Conceptual framework of a digital twin to evaluate the degradation status of complex engineering systems[J].Procedia CIRP, 2019, 86: 61-67. DOI:10.1016/j.procir.2020.01.043.
[18] Wu C L, Zhou Y C, Pereia Pess�F4;a M V, et al. Conceptual digital twin modeling based on an integrated five-dimensional framework and TRIZ function model[J].Journal of Manufacturing Systems, 2021, 58: 79-93. DOI:10.1016/j.jmsy.2020.07.006.
[19] O’Sullivan J, O’Sullivan D, Bruton K. A case-study in the introduction of a digital twin in a large-scale smart manufacturing facility[J].Procedia Manufacturing, 2020, 51: 1523-1530. DOI:10.1016/j.promfg.2020.10.212.
[20] Yildiz E, M�F8;ller C, Bilberg A. Virtual factory: Digital twin based integrated factory simulations[J]. Procedia CIRP, 2020, 93: 216-221. DOI:10.1016/j.procir.2020.04.043.
[21] Liu L Y, Du H X, Wang H F, et al. Construction and application of digital twin system in work- shop production process[J]. Computer Integrated Manufacturing System, 2019, 25(6):1536-1545. DOI:10.13196/j.cims.2019.06.021. (in Chinese)
[22] Schroeder G N, Steinmetz C, Pereira C E, et al. Digital twin data modeling with AutomationML and a communication methodology for data exchange[J].IFAC-PapersOnLine, 2016, 49(30): 12-17. DOI:10.1016/j.ifacol.2016.11.115.
[23] Zhu Z X, Liu C, Xu X. Visualisation of the Digital Twin data in manufacturing by using augmented reality[J]. Procedia CIRP, 2019, 81: 898-903. DOI:10.1016/j.procir.2019.03.223.
[24] Qiu S G, Liu S T, Kong D S, et al. Three-dimensional virtual-real mapping of aircraft automatic spray operation and online simulation monitoring[J]. Virtual Reality & Intelligent Hardware, 2019, 1(6): 611-621. DOI:10.1016/j.vrih.2019.10.003.
[25] Oyekan J O, Hutabarat W, Tiwari A, et al. The effectiveness of VMs in developing collaborative strategies between industrial robots and humans[J]. Robotics and Computer-Integrated Manufacturing, 2019, 55: 41-54. DOI:10.1016/j.rcim.2018.07.006.
[26] Zhuang C B, Miao T, Liu J H, et al. The connotation of digital twin, and the construction and application method of shop-floor digital twin[J]. Robotics and Computer-Integrated Manufacturing, 2021, 68: 102075. DOI:10.1016/j.rcim.2020.102075.
[27] Falah M F, Sukaridhoto S, Al Rasyid M U H, et al. Design of virtual engineering and digital twin platform as implementation of cyber-physical systems[J]. Procedia Manufacturing, 2020, 52: 331-336. DOI:10.1016/j.promfg.2020.11.055.
[28] Liu S M, Bao J S, Lu Y Q, et al. Digital twin modeling method based on biomimicry for machining aerospace components[J]. Journal of Manufacturing Systems, 2021, 58: 180-195. DOI:10.1016/j.jmsy.2020.04.014.
[29] Hutabarat W, Oyekan J, Turner C, et al. Combining virtual reality enabled simulation with 3D scanning technologies towards smart manufacturing[C]//2016 Winter Simulation Conference(WSC). Washington, DC, USA, 2016: 2774-2785. DOI:10.1109/WSC.2016.7822314.
[30] Li Z, Wang H F, Liu T T, et al. Design of workshop real-time monitoring system for manufacturing process[J]. Mechanical Design and Manufacturing, 2013, 3: 256-259. DOI:10.19356/j.cnki.1001-3997.2013.03.080. (in Chinese)
[31] Cao W, Jiang P Y, Jiang K Y, et al. Real-time data collection and visual monitoring method for discrete manufacturing workshop based on RFID technology [J]. Computer Integrated Manufacturing System, 2017, 23(2):273-284. DOI:10.13196/j.cims.201 7.02.006. (in Chinese)
[32] Huang Y, Williams B C, Zheng L. Reactive, model-based monitoring in RFID-enabled manufacturing[J]. Computers in Industry, 2011, 62(8/9): 811-819. DOI:10.1016/j.compind.2011.08.003.
[33] Zong X Y, Luan Y, Wang H L, et al. A multi-robot monitoring system based on digital twin[J]. Procedia Computer Science, 2021, 183: 94-99. DOI:10.1016/j.procs.2021.02.035.
[34] Zhao H R, Liu J H, Xiong H, et al. Three-dimensional visualization real-time monitoring method for digital twin workshop[J]. Computer Integrated Manufacturing System, 2019, 25(6): 1432-1443. DOI:10.13196/j.cims.2019.06.011. (in Chinese)
[35] Jiang K, Ke R, Zhao X Y, et al. Research on virtual monitoring system of digital workshop[J]. Aviation Manufacturing Technology, 2016, 20:97-100, 104. DOI:10.160 80/j.issn1671-833x.2016.20.097. (in Chinese)
[36] Zhou C, Sun K T, Li J, et al. Workshop 3D visual monitoring system based on digital twin [J/OL].Computer Integrated Manufacturing System.(2020-08-17)[2021-05-15].http://kns.cnki.net/kcms/detail/11.5946.TP.20200817.0917.008.html.(in Chinese)
[37] Li H, Liu G, Wen X Y, Wang H Q, et al. Industrial safety control system and key technology of digital twin system for human-computer interaction [J]. Computer integration Manufacturing System, 2021, 27(2): 374-389. DOI:10.13196/j.cims.2021.02.006. (in Chinese)
[38] Qiu C, Zhou S E, Liu Z Y, et al. Digital assembly technology based on augmented reality and digital twins: A review[J]. Virtual Reality & Intelligent Hardware, 2019, 1(6): 597-610. DOI:10.1016/j.vrih.2019.10.002.
[39] Ma X, Tao F, Zhang M, et al. Digital twin enhanced human-machine interaction in product lifecycle[J]. Procedia CIRP, 2019, 83: 789-793. DOI:10.1016/j.procir.2019.04.330.
[40] Ke S Q, Xiang F, Zhang Z, et al. A enhanced interaction framework based on VR, AR and MR in digital twin[J]. Procedia CIRP, 2019, 83: 753-758. DOI:10.1016/j.procir.2019.04.103.
[41] Shuai C L, Chen X M, Qiu S G. Application thinking and prospects of virtual reality technology in aviation intelligent manufacturing[J]. Aviation Manufacturing Technology, 2016, 16: 26-33. DOI:10.16080/j.issn1671-833x.2016.16.026. (in Chinese)
[42] Tao F, Liu W R, Liu J H, etc. Digital twin and its application exploration[J]. Computer Integrated Manufacturing System, 2018, 24(1):1-18. DOI:10.13196/j.cims.201 8.01.001. (in Chinese)
[43] Chakraborty B K, Sarma D, Bhuyan M K, et al. Review of constraints on vision-based gesture recognition for human-computer interaction[J]. IET Computer Vision, 2018, 12(1): 3-15. DOI:10.1049/iet-cvi.2017.0052.
[44] Liu Y, Zhao X. Research on industrial digital twin technology system and key technologies [J]. Information and Communication Technology and Policy, 2021, 47(1): 8-13.(in Chinese)

Memo

Memo:

Biography: Zhang Qinglei(1973—), male, doctor, professor, qlzhang@shmtu.edu.cn.
Foundation items: The National Natural Science Foundation of China(No. 51875332), the Capacity Building Projects of Some Local Universities of Shanghai Science and Technology Commission(No. 18040501600).
Citation: Zhang Qinglei, Yang Zhiwei, Duan Jianguo, et al.Three-dimensional visualization interactive system for digital twin workshop[J].Journal of Southeast University(English Edition), 2021, 37(2):137-152.DOI:10.3969/j.issn.1003-7985.2021.02.003.

Common functions