Adversarial Reinforcement Learning-Based Converged Communication Efficiency Improvement Method for Power Distribution Network

doi:10.11930/j.issn.1004-9649.202210068

Abstract

Abstract: In order to satisfy the diversified communication requirements of terminal source nodes in power distribution network, it is necessary to optimize the communication orchestration in power distribution unified communication network. Firstly, we construct the joint optimization problem of data transmission delay and energy consumption. Then, the joint optimization problem is modeled as a multi-armed bandit problem, and an adversarial reinforcement learning-based communication orchestration algorithm for power distribution unified communication network is proposed, which uses the historical orchestration information and the perceived adversary between source nodes to dynamically learn the communication orchestration strategy. Finally, the superior performance of the proposed algorithm is verified through simulation.

Key words: distribution network, reinforcement learning, adversary awareness, communication orchestration

PENG Linyu, LIU Xu, TANG Wei, LIU Qing, FANG Hao, ZHANG Guanghui. Adversarial Reinforcement Learning-Based Converged Communication Efficiency Improvement Method for Power Distribution Network[J]. Electric Power, 2023, 56(9): 127-133.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.electricpower.com.cn/EN/10.11930/j.issn.1004-9649.202210068

https://www.electricpower.com.cn/EN/Y2023/V56/I9/127

References

[1] 王成山, 李鹏. 分布式发电、微网与智能配电网的发展与挑战[J]. 电力系统自动化, 2010, 34(2): 10–14, 23
WANG Chengshan, LI Peng. Development and challenges of distributed generation, the micro-grid and smart distribution system[J]. Automation of Electric Power Systems, 2010, 34(2): 10–14, 23
[2] 蔡乾, 王晶, 耿天翔, 等. 考虑新能源资源及出力特性的全局备用容量优化方法[J]. 中国电力, 2021, 54(2): 90–97
CAI Qian, WANG Jing, GENG Tianxiang, et al. Global reserve optimization method considering resources and output characteristics of renewable energy[J]. Electric Power, 2021, 54(2): 90–97
[3] 方如举, 葛瑜, 孙伟, 等. 基于WSNs的智能配电网通信数据传输带宽的优化分配策略[J]. 电力系统保护与控制, 2021, 49(23): 88–95
FANG Ruju, GE Yu, SUN Wei, et al. Transmission bandwidth optimal allocation strategy of communication data for a smart distribution grid based on WSNs[J]. Power System Protection and Control, 2021, 49(23): 88–95
[4] 李沛哲, 肖振锋, 陈仲伟, 等. 电力终端通信接入网通信技术匹配[J]. 电力科学与技术学报, 2021, 36(3): 125–134
LI Peizhe, XIAO Zhenfeng, CHEN Zhongwei, et al. Analysis of communication matching technology of power terminal communication access network[J]. Journal of Electric Power Science and Technology, 2021, 36(3): 125–134
[5] 杨振铨, 项基, 李艳君. 配合主网调度的配电网分布式电源主动控制策略[J]. 中国电机工程学报, 2019, 39(11): 3176–3186
YANG Zhenquan, XIANG Ji, LI Yanjun. Active control strategy of distributed generations for utility grid cooperation in distribution network[J]. Proceedings of the CSEE, 2019, 39(11): 3176–3186
[6] LIU J M, ZHAO Z Y, JI J, et al. Research and application of wireless sensor network technology in power transmission and distribution system[J]. Intelligent and Converged Networks, 2020, 1(2): 199–220.
[7] TANG Y J, CHENG N, WU W, et al. Delay-minimization routing for heterogeneous VANETs with machine learning based mobility prediction[J]. IEEE Transactions on Vehicular Technology, 2019, 68(4): 3967–3979.
[8] APRUZZESE G, ANDREOLINI M, MARCHETTI M, et al. Deep reinforcement adversarial learning against botnet evasion attacks[J]. IEEE Transactions on Network and Service Management, 2020, 17(4): 1975–1987.
[9] 吕昊, 何益鸣, 田浩, 等. 基于物联网的园区综合能源系统快速通信网络建模与仿真[J]. 中国电力, 2022, 55(5): 166–173
LV Hao, HE Yiming, TIAN Hao, et al. Modeling and simulation of fast communication network for park integrated energy system based on IoT[J]. Electric Power, 2022, 55(5): 166–173
[10] LIAO H J, ZHOU Z Y, MUMTAZ S, et al. Robust task offloading for IoT fog computing under information asymmetry and information uncertainty[C]//ICC 2019 - 2019 IEEE International Conference on Communications (ICC). Shanghai, China. IEEE, 2019: 1–6.
[11] 陶维青, 窦开明, 方陈, 等. 配电网相量数据接入方式比较及技术分析[J]. 电网技术, 2019, 43(3): 784–792
TAO Weiqing, DOU Kaiming, FANG Chen, et al. Comparison and technical analysis of phasor data access modes in distribution network[J]. Power System Technology, 2019, 43(3): 784–792
[12] 王玉东, 刘伟, 刘洁, 等. P-OTN技术在电力骨干通信网的应用及多业务承载策略研究[J]. 中国电力, 2017, 50(10): 28–34
WANG Yudong, LIU Wei, LIU Jie, et al. Research on application of P-OTN technology in electric power backbone communication network and multi-service bearing strategy[J]. Electric Power, 2017, 50(10): 28–34
[13] 梁云, 黄莉, 侯兴哲, 等. 配电网CPS的通信需求和网络资源分配方案探讨[J]. 中国电力, 2019, 52(1): 32–39
LIANG Yun, HUANG Li, HOU Xingzhe, et al. Discussion on communication requirement and network resource allocation scheme of cyber physical system for distribution network[J]. Electric Power, 2019, 52(1): 32–39
[14] BANAGAR M, MAHAM B, POPOVSKI P, et al. Power distribution of device-to-device communications in underlaid cellular networks[J]. IEEE Wireless Communications Letters, 2016, 5(2): 204–207.
[15] 王建平, 王梦彪, 王金玲, 等. 智能配电网通信实时性与可靠性QoS路由机制研究[J]. 电子测量与仪器学报, 2013, 27(3): 187–193
WANG Jianping, WANG Mengbiao, WANG Jinling, et al. Research on real-time and reliability of intelligent distribution networks QoS routing mechanism[J]. Journal of Electronic Measurement and Instrument, 2013, 27(3): 187–193
[16] SUN W, YUAN X J, WANG J P, et al. Quality of service networking for smart grid distribution monitoring[C]//2010 First IEEE International Conference on Smart Grid Communications. Gaithersburg, MD, USA. IEEE, 2010: 373–378.
[17] MOUSTAPHA A I, SELMIC R R. Wireless sensor network modeling using modified recurrent neural networks: application to fault detection[J]. IEEE Transactions on Instrumentation and Measurement, 2008, 57(5): 981–988.
[18] DHURANDHER S K, MISRA S, OBAIDAT M S, et al. An ant colony optimization approach for reputation and quality-of-service-based security in wireless sensor networks[J]. Security and Communication Networks, 2009, 2(2): 215–224.
[19] ZHOU Z Y, CHEN X Y, LIAO H J, et al. Context-aware learning-based resource allocation for ubiquitous power IoT[J]. IEEE Internet of Things Magazine, 2020, 3(4): 46–52.
[20] 王毅, 王智微, 何新. 智能电站数据中台建设与应用[J]. 中国电力, 2021, 54(3): 61–67, 176
WANG Yi, WANG Zhiwei, HE Xin. Data middle platform construction and application of intelligent power stations[J]. Electric Power, 2021, 54(3): 61–67, 176
[21] HONG A, LUNSCHER N, HU T H, et al. A multimodal emotional human–robot interaction architecture for social robots engaged in bidirectional communication[J]. IEEE Transactions on Cybernetics, 2021, 51(12): 5954–5968.
[22] ZHANG S X, LIAO H J, ZHOU Z Y, et al. Federated deep actor-critic-based task offloading in air-ground electricity IoT[C]//2021 IEEE Global Communications Conference (GLOBECOM). Madrid, Spain. IEEE, 2022: 1–6.
[23] WANG Z, JIA Z H, LIAO H J, et al. Energy-aware and URLLC-aware task offloading for Internet of health things[C]//GLOBECOM 2020-2020 IEEE Global Communications Conference. Taipei, China. IEEE, 2021: 1–6.
[24] JIA Z H, LIAO H J, ZHOU Z Y, et al. Multi-dimension resource allocation for NOMA-edge computing-based 6G power IoT[C]//2021 IEEE International Conference on Communications Workshops (ICC Workshops). Montreal, QC, Canada. IEEE, 2021: 1–6.
[25] LIAO H J, ZHOU Z Y, ZHAO X W, et al. Task offloading for vehicular fog computing under information uncertainty: a matching-learning approach[C]//2019 15 th International Wireless Communications & Mobile Computing Conference (IWCMC). Tangier, Morocco. IEEE, 2019: 2001–2006.