Routing Algorithm for Power Communication Networks Based on Serivce Differentiated Transmission Requirements

doi:10.11930/j.issn.1004-9649.202405020

Abstract

Abstract:

The electric power communication network, pivotal in ensuring the stable operation of the power grid, is tasked with transmitting control instructions and collecting status data. Addressing the intelligent routing challenge within the electric power communication network under multiple constraints, we propose an innovative routing algorithm that seamlessly integrates Message Passing Neural Network (MPNN) with deep reinforcement learning algorithms. Implemented through the TensorFlow framework, this algorithm has been rigorously validated in a simulation environment constructed using OpenAI Gym. After undergoing over 8,000 training iterations, the algorithm demonstrates remarkable performance enhancements, outperforming traditional shortest path and load balancing algorithms in terms of routing selection capabilities. Furthermore, it has exhibited robust adaptability and resilience in generalization tests on new topology maps and link failure simulation experiments.

Key words: power communication network, routing optimization, message passing neural network, deep reinforcement learning, multiple constraints

Songping XUE, Dequan GAO, Ziyan ZHAO, Yuqian LIN, Zejing GUANG, Dawei ZHANG. Routing Algorithm for Power Communication Networks Based on Serivce Differentiated Transmission Requirements[J]. Electric Power, 2024, 57(11): 183-190.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.electricpower.com.cn/EN/10.11930/j.issn.1004-9649.202405020

https://www.electricpower.com.cn/EN/Y2024/V57/I11/183

Figures/Tables 21

Fig.1 Topology of 500 kV backbone network in Fujian Province

Table 1 Normalized reward for the three services

业务名称		奖励值
Ⅰ类生产控制业务		0.125
Ⅱ类生产控制业务		0.500
Ⅲ类生产控制业务		1.000

Fig.2 Example of service transmission

Table 2 Reliability requirements for different services

业务名称		可靠性要求
Ⅰ类生产控制业务		0.90
Ⅱ类生产控制业务		0.88
Ⅲ类生产控制业务		0.86

Table 3 Transmission distance limitations for different services

业务名称		传输距离限制/km
Ⅰ类生产控制业务		1000
Ⅱ类生产控制业务		950
Ⅲ类生产控制业务		900

Fig.3 Agent interacting with the power communication network

Fig.4 Architecture of message passing neural network

Fig.5 Algorithm 1 message passing neural network

Fig.6 Workflow of deep reinforcement learning algorithm

Fig.8 Loss curve

Table 4 Table of hyperparameters during training

超参数名称		超参数值
链路隐藏状态link_state_dim		32
学习率		0.0006
消息传递次数T		4
读出层层数readout_layers		3
读出层神经元个数readout_units		35
一次读出的数据个数batch_size		32
丢弃层丢弃概率dropout_rate		0.01

Fig.7 Algorithm 2 Deep Reinforcement Learning Algorithm

Fig.9 Score curve

Fig.10 Algorithm evaluation in the initial environment

Fig.11 Nsfnet topology

Fig.12 Algorithms evaluation under Nsfnet

Fig.13 Random topology

Fig.14 Algorithm evaluation under random topology

Fig.15 Algorithm evaluation without failures

Fig.16 Algorithm evaluation under one link failure

Fig.17 Algorithm evaluation under two link failures

References 25

1	赵子岩, 刘建明. 基于业务风险均衡度的电力通信网可靠性评估算法[J]. 电网技术, 2011, 35 (10): 209- 213.
	ZHAO Ziyan, LIU Jianming. A new service risk balancing based method to evaluate reliability of electric power communication network[J]. Power System Technology, 2011, 35 (10): 209- 213.
2	夏超鹏. 泛在电力物联网在电力市场应用中的展望[J]. 发电技术, 2020, 41 (2): 142- 149.
	XIA Chaopeng. Prospect of ubiquitous power Internet of Things in electricity market[J]. Power Generation Technology, 2020, 41 (2): 142- 149.
3	王勇, 张明, 马洲俊, 等. 电力线信道通信特性影响因素分析[J]. 电力科学与技术学报, 2021, 36 (3): 157- 164, 173.
	WANG Yong, ZHANG Ming, MA Zhoujun, et al. Analysis of the factors influencing the communication characteristics of power line channel[J]. Journal of Electric Power Science and Technology, 2021, 36 (3): 157- 164, 173.
4	王志强, 吴庆, 张拯, 等. 基于异常分析的电力信息通信系统运维策略[J]. 陕西电力, 2016, 44 (4): 84- 87.
	WANG Zhiqiang, WU Qing, ZHANG Zheng, et al. Maintenance strategy of power grid information and telecommunication system based on abnormal analysis[J]. Shanxi Electric Power, 2016, 44 (4): 84- 87.
5	皮志勇, 朱益, 廖玄, 等. 基于深度学习的智能变电站通信链路故障定位方法[J]. 中国电力, 2023, 56 (7): 136- 145.
	PI Zhiyong, ZHU Yi, LIAO Xuan, et al. Fault location method for communication link in smart substation based on deep learning[J]. Electric Power, 2023, 56 (7): 136- 145.
6	皮志勇, 朱益, 廖玄, 等. 智能变电站多信息融合建模的通信链路故障定位方法[J]. 中国电力, 2023, 56 (8): 207- 215.
	PI Zhiyong, ZHU Yi, LIAO Xuan, et al. Fault location method for communication link with multi-information fusion modeling of smart substation[J]. Electric Power, 2023, 56 (8): 207- 215.
7	何天玲, 蔡立波. 耦合电网业务约束条件的电力通信故障检修辅助决策的实现[J/OL]. 中国电力, 2024: 1–8. (2024-03-21). https://kns.cnki.net/kcms/detail/11.3265.tm.20240320.1018.002.html.
	HE Tianling, CAI Libo. Implementation of power communication fault and maintenance aided decision-making coupled with power network service constraints[J/OL]. Electric Power, 2024: 1–8. (2024-03-21). https://kns.cnki.net/kcms/detail/11.3265.tm.20240320.1018.002.html.
8	李彬, 卢超, 景栋盛, 等. 负载与风险联合均衡的电力通信网路由优化算法[J]. 中国电机工程学报, 2019, 39 (9): 2713- 2722.
	LI Bin, LU Chao, JING Dongsheng, et al. An optimized routing algorithm with load and risk joint balance in electric communication network[J]. Proceedings of the CSEE, 2019, 39 (9): 2713- 2722.
9	崔力民, 孙静月, 李珊君, 等. 一种基于熵的电力通信网络业务资源均匀分配算法[J]. 电网技术, 2017, 41 (9): 3066- 3073.
	CUI Limin, SUN Jingyue, LI Shanjun, et al. An algorithm for business resource uniform distribution in power communication network based on entropy[J]. Power System Technology, 2017, 41 (9): 3066- 3073.
10	顿聪颖, 金凤林, 谭诗翰, 等. 基于负载均衡的低轨卫星网络动态路由算法[J]. 无线电工程, 2022, 52 (5): 790- 798.
	DUN Congying, JIN Fenglin, TAN Shihan, et al. Dynamic routing algorithm for low orbit satellite network based on load balancing[J]. Radio Engineering, 2022, 52 (5): 790- 798.
11	孙严智, 刘旋, 刘宇明, 等. 遗传算法电力通信网关键业务备选路由配置[J]. 云南电力技术, 2017, 45 (1): 113- 117.
	SUN Yanzhi, LIU Xuan, LIU Yuming, et al. Research on key service backup routing method based on genetic algorithm in power communication network[J]. Yunnan Electric Power, 2017, 45 (1): 113- 117.
12	王素红, 唐煜星, 郭文豪, 等. 考虑优先级的智能电网业务调度与资源分配方案[J]. 南方电网技术, 2024, 18 (4): 59- 70, 79.
	WANG Suhong, TANG Yuxing, GUO Wenhao, et al. Service scheduling and resource allocation scheme of smart grid considering priority[J]. Southern Power System Technology, 2024, 18 (4): 59- 70, 79.
13	刘思放. 电力需求响应业务传输机制及路由优化策略研究[D]. 北京: 华北电力大学, 2020.
	LIU Sifang. Research on transmission mechanism and routing optimized strategy for power demand response service[D]. Beijing: North China Electric Power University, 2020.
14	曾瑛, 李伟坚, 陈媛媛, 等. 基于业务优先级的电力调度数据网拥塞规避算法[J]. 电力系统保护与控制, 2014, 42 (2): 49- 55.
	ZENG Ying, LI Weijian, CHEN Yuanyuan, et al. A congestion avoidance algorithm based on the service priority for electric power dispatching data network[J]. Power System Protection and Control, 2014, 42 (2): 49- 55.
15	ALMASAN P, SUÁREZ-VARELA J, RUSEK K, et al. Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case[J]. Computer Communications, 2022, 196, 184- 194.
16	YU C H, LAN J L, GUO Z H, et al. DROM: optimizing the routing in software-defined networks with deep reinforcement learning[J]. IEEE Access, 2018, 6, 64533- 64539.
17	HARTERT R, VISSICCHIO S, SCHAUS P, et al. A declarative and expressive approach to control forwarding paths in carrier-grade networks[J]. ACM SIGCOMM Computer Communication Review, 2015, 45 (4): 15- 28.
18	FRANCOIS P, FILSFILS C, EVANS J, et al. Achieving sub-second IGP convergence in large IP networks[J]. ACM SIGCOMM Computer Communication Review, 2005, 35 (3): 35- 44.
19	JAIN S, KUMAR A, MANDAL S, et al. B4: Experience with a globally-deployed software defined WAN[J]. ACM SIGCOMM Computer Communication Review, 2013, 43 (4): 3- 14.
20	何天玲. 基于业务N-2的电力通信网可靠性评估方法研究[J]. 电力信息与通信技术, 2021, 19 (2): 36- 43.
	HE Tianling. Research on reliability evaluation method of power communication network based on service N-2[J]. Electric Power Information and Communication Technology, 2021, 19 (2): 36- 43.
21	王铮澄, 周艳真, 郭庆来, 等. 考虑电力系统拓扑变化的消息传递图神经网络暂态稳定评估[J]. 中国电机工程学报, 2021, 41 (7): 2341- 2349.
	WANG Zhengcheng, ZHOU Yanzhen, GUO Qinglai, et al. Transient stability assessment of power system considering topological change: a message passing neural network-based approach[J]. Proceedings of the CSEE, 2021, 41 (7): 2341- 2349.
22	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41 (1): 1- 27.
	LIU Quan, ZHAI Jianwei, ZHANG Zongchang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41 (1): 1- 27.
23	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning[J]. ArXiv e-Prints, 2013: arXiv: 1312.5602.
24	ABADI M, BARHAM P, CHEN J, et al. TensorFlow: a system for large-scale machine learning[C]//12th USENIX symposium on operating systems design and implementation (OSDI 16). 2016: 265–283.
25	HEI X J, ZHANG J, BENSAOU B, et al. Wavelength converter placement in least-load-routing-based optical networks using genetic algorithms[J]. Journal of Optical Networking, 2004, 3 (5): 363.

[1]	WANG Li, JIANG Yuxiang, ZENG Xiangjun, ZHAO Bin, LI Junhao. Secondary Frequency Control of Islanded Microgrid Based on Deep Reinforcement Learning [J]. Electric Power, 2025, 58(5): 176-188.
[2]	FAN Huicong, DUAN Zhiguo, CHEN Zhiyong, ZHU Shijia, LIU Hang, LI Wenxiao, YANG Yang. Two-layer Optimization Scheduling for Off-grid Microgrids Based on Multi-agent Deep Policy Gradient [J]. Electric Power, 2025, 58(5): 11-20, 32.
[3]	WANG Guanchao, HUO Yuchong, LI Qun, LI Qiang. Power Optimization of Wind Farms Based on Improved Jensen Model and Deep Reinforcement Learning [J]. Electric Power, 2025, 58(4): 78-89.
[4]	Yi ZENG, Yi ZHOU, Jixiang LU, Liangcai ZHOU, Ningkai TANG, Hong LI. Voltage Control Based on Multi-Agent Safe Deep Reinforcement Learning [J]. Electric Power, 2025, 58(2): 111-117.
[5]	Tianling HE, Libo CAI. Implementation of Power Communication Faults and Maintenance Aided Decision-Making Coupled with Power Network Service Constraints [J]. Electric Power, 2024, 57(12): 178-187.
[6]	Chao ZHANG, Dongmei ZHAO, Yu JI, Ying ZHANG. Real Time Optimal Dispatch of Virtual Power Plant Based on Improved Deep Q Network [J]. Electric Power, 2024, 57(1): 91-100.
[7]	SHI Wenzhe, LI Bingjie, YOU Peipei, ZHANG Ling. Optimization Strategy of Building Energy System Based on Deep Reinforcement Learning [J]. Electric Power, 2023, 56(6): 114-122.
[8]	SONG Weiye, LIU Lingyue, YAN Jie, WANG Hangyu, HE Shukai, HAN Shuang, WANG Minghui, LIU Yongqian. Self-evolving Power Smooth Control Method for Offshore Wind Power Cluster Based On Deep Reinforcement Learning [J]. Electric Power, 2023, 56(3): 36-46.
[9]	ZHANG Shuxing, MA Chi, YANG Zhixue, WANG Yao, WU Hao, REN Zhouyang. Deep Deterministic Policy Gradient Algorithm Based Wind-Photovoltaic-Storage Hybrid System Joint Dispatch [J]. Electric Power, 2023, 56(2): 68-76.
[10]	ZHOU Qingfeng, WANG Sichun, LI Dexin, LIU Jiaqi, LI Tong. A Knowledge Learning Model for Intelligent Check of Wind Farm Dynamic Parameters Based on DDPG [J]. Electric Power, 2022, 55(5): 32-38.
[11]	YANG Pengpeng, WANG Beibei, XU Peng, WANG Gaoqin, ZHENG Yaxian. Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition [J]. Electric Power, 2021, 54(11): 47-58.
[12]	JIA Huibin, GAI Yonghe, LI Baogang, ZHENG Hongda. Power Communication Network Recovery from Large-Scale Failures Based on Reinforcement Learning [J]. Electric Power, 2020, 53(6): 34-40.
[13]	WANG Yang, GAO Hanxing, ZHOU Shengping, WANG Zhihui, HE Yujun. Critical Node Identification for Electric Power Communication Network Based on Characteristics of Topology and Services [J]. Electric Power, 2018, 51(10): 111-118.
[14]	XIE Yingjun, WANG Yuting, LI Wei, DING Huixia, WANG Zhihui, ZHANG Shixian, ZHOU Ruankun. Indicator System for Operation Quality of Electric Power Communication Network [J]. Electric Power, 2017, 50(10): 22-27.
[15]	CAO Yuan, ZHAO Yongli, YU Xiaosong, ZHANG Jie. Secure Power Communication Network Architecture Driven by Quantum Key Distribution [J]. Electric Power, 2017, 50(10): 8-11.