[1] Wang Zhiwei, Wang Junbo, Yang Fan, Lin Min, et al. Q-learning-based energy transmission schedulingover a fading channel [J]. Journal of Southeast University (English Edition), 2020, 36 (4): 393-398. [doi:10.3969/j.issn.1003-7985.2020.04.004]

Copy

Q-learning-based energy transmission schedulingover a fading channel()

Share：

Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]

Volumn:

36

Issue:

2020 4

Page:

393-398

Research Field:

Traffic and Transportation Engineering

Publishing date:

2020-12-20

Info

Title:

Q-learning-based energy transmission schedulingover a fading channel

Author(s):

Wang Zhiwei¹;  Wang Junbo²;  Yang Fan²;  Lin Min³

¹ School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China
² School of Information Science and Engineering, Southeast University, Nanjing 210096, China
³ School of Science, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

Keywords:

energy harvesting;  channel state information;  Q-learning;  transmission scheduling

PACS:

U491

DOI:

10.3969/j.issn.1003-7985.2020.04.004

Abstract:

To solve the problem of energy transmission in the Internet of Things(IoTs), an energy transmission schedule over a Rayleigh fading channel in the energy harvesting system(EHS)with a dedicated energy source(ES)is considered. According to the channel state information(CSI)and the battery state, the charging duration of the battery is determined to jointly minimize the energy consumption of ES, the battery’s deficit charges and overcharges during energy transmission. Then, the joint optimization problem is formulated using the weighted sum method. Using the ideas from the Q-learning algorithm, a Q-learning-based energy scheduling algorithm is proposed to solve this problem. Then, the Q-learning-based energy scheduling algorithm is compared with a constant strategy and an on-demand dynamic strategy in energy consumption, the battery’s deficit charges and the battery’s overcharges. The simulation results show that the proposed Q-learning-based energy scheduling algorithm can effectively improve the system stability in terms of the battery’s deficit charges and overcharges.

References:

[1] Adila A S, Husam A, Husi G. Towards the self-powered internet of things(IoT)by energy harvesting: Trends and technologies for green IoT[C]// 2018 2nd International Symposium on Small-scale Intelligent Manufacturing Systems(SIMS). Cavan, Ireland, 2018: 1-5.
[2] Kamalinejad P, Mahapatra C, Sheng Z, et al. Wireless energy harvesting for the internet of things[J]. IEEE Communications Magazine, 2015, 53(6): 102-108. DOI: 10.1109/MCOM.2015.7120024.
[3] Luo Y, Pu L N, Zhao Y X, et al. DTER: Optimal two-step dual tunnel energy requesting for RF-based energy harvesting system[J].IEEE Internet of Things Journal, 2018, 5(4): 2768-2780. DOI:10.1109/jiot.2018.2813429.
[4] Kansal A, Hsu J, Zahedi S, et al. Power management in energy harvesting sensor networks[J]. ACM Transactions on Embedded Computing Systems, 2007, 6(4): 32. DOI:10.1145/1274858.1274870.
[5] Hsu R C, Liu C T, Wang H L. A reinforcement learning-based ToD provisioning dynamic power management for sustainable operation of energy harvesting wireless sensor node[J].IEEE Transactions on Emerging Topics in Computing, 2014, 2(2): 181-191. DOI:10.1109/tetc.2014.2316518.
[6] Hsu R C, Lin T. A fuzzy Q-learning based power management for energy harvest wireless sensor node[C]// 2018 Int Conf HPCS. Orléans, France, 2018: 957-961.
[7] Mastronarde N, van der Schaar M. Joint physical-layer and system-level power management for delay-sensitive wireless communications[J].IEEE Transactions on Mobile Computing, 2013, 12(4): 694-709. DOI:10.1109/tmc.2012.36.
[8] Ertel R B, Reed J H. Generation of two equal power correlated Rayleigh fading envelopes[J].IEEE Communications Letters, 1998, 2(10): 276-278. DOI:10.1109/4234.725222.
[9] Wang J B, Feng M, Song X Y, et al. Imperfect CSI based joint bit loading and power allocation for deadline constrained transmission[J].IEEE Communications Letters, 2013, 17(5): 826-829. DOI:10.1109/lcomm.2013.031313.122583.
[10] Luo Y, Pu L, Zhao Y, et al. Optimal energy requesting strategy for rf-based energy harvesting wireless communications[C]// IEEE Conf Comput Commun. Atlanta, GA, USA, 2017: 1-9.
[11] Liu G P, Whidborne J F, Yang J B, et al. Multiobjective optimisation and control[J]. Wetlands Ecology & Management, 2003, 17(2):157-164. DOI:10.1007/s11273-008-9090-x.
[12] Mitchell T M, Carbonell J G, Michalski R S. Machine learning[M]. Boston, MA, USA: Springer, 1986:39-42.

Memo

Memo:

Biographies: Wang Zhiwei(1990—), male, doctor; Wang Junbo(corresponding author), male, doctor, professor, jbwang@seu.edu.cn.
Foundation item: The National Natural Science Foundation of China(No. 51608115).
Citation: Wang Zhiwei, Wang Junbo, Yang Fan, et al. Q-learning-based energy transmission scheduling over a fading channel[J].Journal of Southeast University(English Edition), 2020, 36(4):393-398.DOI:10.3969/j.issn.1003-7985.2020.04.004.

Common functions