[1] Fang Yunan, Jiang Rui,. Gradient parameter data integrity protection schemein the Ring Allreduce architecture [J]. Journal of Southeast University (English Edition), 2023, 39 (1): 81-88. [doi:10.3969/j.issn.1003-7985.2023.01.010]
Copy
Gradient parameter data integrity protection schemein the Ring Allreduce architecture(
)
Ring Allreduce中的梯度参数完整性保护方案
)Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]
- Volumn:
- 39
- Issue:
- 2023 1
- Page:
- 81-88
- Research Field:
- Computer Science and Engineering
- Publishing date:
- 2023-03-20
Info
- Title:
- Gradient parameter data integrity protection schemein the Ring Allreduce architecture
- Ring Allreduce中的梯度参数完整性保护方案
- Author(s):
- Fang Yunan; Jiang Rui
- School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China
- Keywords:
- distributed machine learning; data integrity; group key agreement; Ring Allreduce architecture
- PACS:
- TP309.2
- DOI:
- 10.3969/j.issn.1003-7985.2023.01.010
- Abstract:
- As there is no research on protecting gradient parameter data integrity in the Ring Allreduce architecture, a Ring Allreduce architecture oriented gradient parameter data integrity protection scheme(RAA-DIP)is proposed. The identity-based group key agreement algorithm and the Boneh-Lynn-Shacham signature are used to protect the integrity of gradient parameter data in Ring Allreduce(RAA). Combined with identity authentication and the key negotiation algorithm, secure and efficient dynamic management of working nodes is realized. On the basis of the decisional bilinear Diffie-Hellman problem, secure group key negotiation is implemented so that the key generation center or network attackers cannot calculate the shared secret of worker nodes, which solves the key escrow problem and ensures the integrity of transmission gradient data. Finally, the RAA-DIP scheme is formally proved, and its simulation performance is compared with those of related schemes. The results show that the RAA-DIP scheme can guarantee the integrity of the gradient parameter data transmission process in Ring Allreduce, realize the dynamic management of working nodes, and solve the problem of key escrow. Compared with related schemes, it can meet security and performance requirements.
- 为了解决Ring Allreduce架构中没有保护梯度参数完整性的问题, 提出了一种面向Ring Allreduce架构的梯度参数完整性保护方案(RAA-DIP).使用基于身份的组密钥协商算法和Boneh-Lynn-Shacham(BLS)签名方案保障Ring Allreduce中梯度参数的完整性.结合身份认证和密钥协商算法, 实现安全高效的动态工作节点管理.基于DBDH问题实现安全的组密钥协商, 使密钥生成中心(KGC)或网络攻击者无法计算工作节点的共享密钥, 解决密钥托管问题, 保障传输参数的完整性.对RAA-DIP方案进行形式化证明, 并将仿真结果与相关方案进行比较.结果表明:RAA-DIP方案可以保障Ring Allreduce中梯度参数传输过程的完整性, 实现工作节点动态管理, 解决密钥托管问题;与其他方案相比, RAA-DIP方案可以同时满足安全和性能需求.
References:
[1] Gibiansky A. Bringing HPC techniques to deep learning [EB/OL].(2017-02-21)[2022-12-05]. https://andrew.gibiansky.com/blog/machine-learning/baidu-allreduce.
[2] Kim S, Yu G I, Park H, et al. Parallax: Sparsity-aware data parallel training of deep neural networks [C]//Proceedings of the Fourteenth EuroSys Conference. Dresden, MO, USA, 2019: 1-15. DOI: 10.1145/3302424.3303957.
[3] Kurth T, Treichler S, Romero J, et al. Exascale deep learning for climate analytics [C]//International Conference for High Performance Computing, Networking, Storage and Analysis. Dallas, TX, USA, 2018:649-660. DOI: 10.1109/SC.2018.00054.
[4] Lü G F, Li M F, An H, et al. Distributed deep learning system for cancerous region detection on Sunway TaihuLight [J].CCF Transactions on High Performance Computing, 2020, 2(4): 348-361. DOI: 10.1007/s42514-020-00046-5.
[5] Cui H, Radosavljevic V, Chou F C, et al. Multimodal trajectory predictions for autonomous driving using deep convolutional networks [C]//2019 International Conference on Robotics and Automation. Montreal, Canada, 2019: 2090-2096. DOI: 10.1109/ICRA.2019.8793868.
[6] Liang J, Makoviychuk V, Handa A, et al. Gpu-accelerated robotic simulation for distributed reinforcement learning [C]//Conference on Robot Learning. Zurich, Switzerland, 2018: 270-282. DOI: abs/1810.05762.
[7] Reisizadeh A, Prakash S, Pedarsani R, et al. Codedreduce: A fast and robust framework for gradient aggregation in distributed learning [J].IEEE/ACM Transactions on Networking, 2022, 30(1): 148-161. DOI: 10.1109/TNET.2021.3109097.
[8] Jia X, Song S, He W, et al. Highly scalable deep learning training system with mixed-precision: Training ImageNet in four minutes [EB/OL].(2018-07-30)[2022-09-05]. https://arxiv.org/abs/1807.11205.
[9] Mikami H, Suganuma H, Tanaka Y, et al. Massively distributed SGD: ImageNet/ResNet-50 training in a flash [EB/OL].(2019-03-05)[2022-09-05]. https://arxiv.org/abs/1811.05233.
[10] Zhang A, Chen J, Hu R Q, et al. SeDS: Secure data sharing strategy for D2D communication in LTE-Advanced networks [J].IEEE Transactions on Vehicular Technology, 2015, 65(4): 2659-2672. DOI: 10.1109/TVT.2015.2416002.
[11] Lopes A P G, Gondim P R L. Group authentication protocol based on aggregated signatures for D2D communication [J].Computer Networks, 2020, 178: 107192. DOI: 10.1016/j.comnet.2020.107192.
[12] Choi K Y, Hwang J Y, Lee D H. Efficient ID-based group key agreement with bilinear maps [C]//Proceedings of the 7th International Workshop on Public Key Cryptography. Singapore, 2004: 130-144. DOI: 10.1007/978-3-540-24632-9_10.
[13] Burmester M, Desmedt Y. A secure and efficient conference key distribution system [C]//EUROCRYPT’94. Perugia, Italy, 1994: 275-286. DOI: 10.1007/BFb0053443.
[14] Kim Y, Perrig A, Tsudik G. Tree-based group key agreement [J].ACM Transactions on Information and System Security, 2004, 7(1): 60-96. DOI: 10.1145/984334.984337.
[15] Mao Y, Sun Y, Wu M, et al. JET: Dynamic join-exit-tree amortization and scheduling for contributory key management [J].IEEE/ACM Transactions on Networking, 2006, 14(5): 1128-1140. DOI: 10.1109/TNET.2006.882851.
[16] Wang M, Yan Z. Privacy-preserving authentication and key agreement protocols for D2D group communications [J].IEEE Transactions on Industrial Informatics, 2017, 14(8): 3637-3647. DOI: 10.1109/TII.2017.2778090.
[17] Scott M. Efficient implementation of cryptographic pairings [EB/OL].(2016-02-08)[2022-12-05].http://www.pairing-conference.org/2007/invited/Scott_slide.pdf.
Memo
- Memo:
-
Biographies: Fang Yunan(1998—), female, graduate; Jiang Rui(corresponding author), male, doctor, professor, R.Jiang@seu.edu.cn.
Foundation items: The National Natural Science Foundation of China(No. 61372103), the Natural Science Foundation of Jiangsu Province(No. BK20201265), Foundation of the National Engineering Research Center of Classified Protection and Safeguard Technology for Cybersecurity(No. C21640-2).
Citation: Fang Yunan, Jiang Rui. Gradient parameter data integrity protection scheme in the Ring Allreduce architecture[J].Journal of Southeast University(English Edition), 2023, 39(1):81-88.DOI:10.3969/j.issn.1003-7985.2023.01.010.