[1] Wang Zhiwei, Wang Junbo, Yang Fan, Lin Min, et al. Q-learning-based energy transmission schedulingover a fading channel [J]. Journal of Southeast University (English Edition), 2020, 36 (4): 393-398. [doi:10.3969/j.issn.1003-7985.2020.04.004]

Copy

Q-learning-based energy transmission schedulingover a fading channel()

衰落信道下基于Q学习的能量调度方案

Share：

Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]

Volumn:

36

Issue:

2020 4

Page:

393-398

Research Field:

Traffic and Transportation Engineering

Publishing date:

2020-12-20

Info

Title:

Q-learning-based energy transmission schedulingover a fading channel

衰落信道下基于Q学习的能量调度方案

Author(s):

Wang Zhiwei¹, Wang Junbo², Yang Fan², Lin Min³

¹ School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China
² School of Information Science and Engineering, Southeast University, Nanjing 210096, China
³ School of Science, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

王志伟¹, 王俊波² , 杨凡² , 林敏³

¹ 东南大学网络空间安全学院, 南京 210096; ² 东南大学信息与工程学院, 南京 210096; ³ 南京邮电大学理学院, 南京 210003

Keywords:

energy harvesting;  channel state information;  Q-learning;  transmission scheduling

能量收集;  信道状态信息;  Q学习;  传输调度

PACS:

U491

DOI:

10.3969/j.issn.1003-7985.2020.04.004

Abstract:

To solve the problem of energy transmission in the Internet of Things(IoTs), an energy transmission schedule over a Rayleigh fading channel in the energy harvesting system(EHS)with a dedicated energy source(ES)is considered. According to the channel state information(CSI)and the battery state, the charging duration of the battery is determined to jointly minimize the energy consumption of ES, the battery’s deficit charges and overcharges during energy transmission. Then, the joint optimization problem is formulated using the weighted sum method. Using the ideas from the Q-learning algorithm, a Q-learning-based energy scheduling algorithm is proposed to solve this problem. Then, the Q-learning-based energy scheduling algorithm is compared with a constant strategy and an on-demand dynamic strategy in energy consumption, the battery’s deficit charges and the battery’s overcharges. The simulation results show that the proposed Q-learning-based energy scheduling algorithm can effectively improve the system stability in terms of the battery’s deficit charges and overcharges.

研究了处于瑞利衰落信道下, 具有一个固定能量源的能量收集系统对能量传输进行调度的问题.根据信道信息和电池的剩余电量状态, 确定电池的充电时长使得能量传输过程中能量源的能量消耗、电池的耗尽次数以及电池电量溢出的次数尽可能最小.接着再使用加权和的方式来表示该优化问题.利用Q学习的思想, 提出了一种基于Q学习的能量调度方案来解决此问题.通过将基于Q学习的能量传输调度方案与2种离线传输策略(静态策略和按需分配的动态策略)在能量消耗、电池电量耗尽次数以及电池电量溢出等方面进行比较, 分析该算法的优势与不足.仿真结果表明, 基于Q学习的能量传输调度方案有效地抑制了电池电量耗尽和电池电量溢出的发生, 从而提高了系统的稳定性.

References:

[1] Adila A S, Husam A, Husi G. Towards the self-powered internet of things(IoT)by energy harvesting: Trends and technologies for green IoT[C]// 2018 2nd International Symposium on Small-scale Intelligent Manufacturing Systems(SIMS). Cavan, Ireland, 2018: 1-5.
[2] Kamalinejad P, Mahapatra C, Sheng Z, et al. Wireless energy harvesting for the internet of things[J]. IEEE Communications Magazine, 2015, 53(6): 102-108. DOI: 10.1109/MCOM.2015.7120024.
[3] Luo Y, Pu L N, Zhao Y X, et al. DTER: Optimal two-step dual tunnel energy requesting for RF-based energy harvesting system[J].IEEE Internet of Things Journal, 2018, 5(4): 2768-2780. DOI:10.1109/jiot.2018.2813429.
[4] Kansal A, Hsu J, Zahedi S, et al. Power management in energy harvesting sensor networks[J]. ACM Transactions on Embedded Computing Systems, 2007, 6(4): 32. DOI:10.1145/1274858.1274870.
[5] Hsu R C, Liu C T, Wang H L. A reinforcement learning-based ToD provisioning dynamic power management for sustainable operation of energy harvesting wireless sensor node[J].IEEE Transactions on Emerging Topics in Computing, 2014, 2(2): 181-191. DOI:10.1109/tetc.2014.2316518.
[6] Hsu R C, Lin T. A fuzzy Q-learning based power management for energy harvest wireless sensor node[C]// 2018 Int Conf HPCS. Orléans, France, 2018: 957-961.
[7] Mastronarde N, van der Schaar M. Joint physical-layer and system-level power management for delay-sensitive wireless communications[J].IEEE Transactions on Mobile Computing, 2013, 12(4): 694-709. DOI:10.1109/tmc.2012.36.
[8] Ertel R B, Reed J H. Generation of two equal power correlated Rayleigh fading envelopes[J].IEEE Communications Letters, 1998, 2(10): 276-278. DOI:10.1109/4234.725222.
[9] Wang J B, Feng M, Song X Y, et al. Imperfect CSI based joint bit loading and power allocation for deadline constrained transmission[J].IEEE Communications Letters, 2013, 17(5): 826-829. DOI:10.1109/lcomm.2013.031313.122583.
[10] Luo Y, Pu L, Zhao Y, et al. Optimal energy requesting strategy for rf-based energy harvesting wireless communications[C]// IEEE Conf Comput Commun. Atlanta, GA, USA, 2017: 1-9.
[11] Liu G P, Whidborne J F, Yang J B, et al. Multiobjective optimisation and control[J]. Wetlands Ecology & Management, 2003, 17(2):157-164. DOI:10.1007/s11273-008-9090-x.
[12] Mitchell T M, Carbonell J G, Michalski R S. Machine learning[M]. Boston, MA, USA: Springer, 1986:39-42.

Memo

Memo:

Biographies: Wang Zhiwei(1990—), male, doctor; Wang Junbo(corresponding author), male, doctor, professor, jbwang@seu.edu.cn.
Foundation item: The National Natural Science Foundation of China(No. 51608115).
Citation: Wang Zhiwei, Wang Junbo, Yang Fan, et al. Q-learning-based energy transmission scheduling over a fading channel[J].Journal of Southeast University(English Edition), 2020, 36(4):393-398.DOI:10.3969/j.issn.1003-7985.2020.04.004.

Common functions