基于业务差异化传输需求下的电力通信网路由算法

doi:10.11930/j.issn.1004-9649.202405020

摘要/Abstract

摘要：

电力通信网负责传递控制指令、收集状态数据，对保障电网的稳定运作至关重要。针对电力通信网络中多约束条件下的智能路由问题，提出了一种结合消息传递神经网络（message passing neural network，MPNN）与深度强化学习算法的智能路由算法。通过Tensor flow框架实现，在Open AI Gym构建的模拟环境进行验证。算法在超过8000次的训练迭代后呈现出显著的性能提升，表现出了较传统最短路径和负载均衡算法更优越的路由选择能力。同时，在新拓扑图的泛化测试和链路故障模拟实验中，也显示出较强的适应性和鲁棒性。

关键词: 电力通信网, 路由优化, 消息神经网络, 深度强化学习, 多约束条件

Abstract:

The electric power communication network, pivotal in ensuring the stable operation of the power grid, is tasked with transmitting control instructions and collecting status data. Addressing the intelligent routing challenge within the electric power communication network under multiple constraints, we propose an innovative routing algorithm that seamlessly integrates Message Passing Neural Network (MPNN) with deep reinforcement learning algorithms. Implemented through the TensorFlow framework, this algorithm has been rigorously validated in a simulation environment constructed using OpenAI Gym. After undergoing over 8,000 training iterations, the algorithm demonstrates remarkable performance enhancements, outperforming traditional shortest path and load balancing algorithms in terms of routing selection capabilities. Furthermore, it has exhibited robust adaptability and resilience in generalization tests on new topology maps and link failure simulation experiments.

Key words: power communication network, routing optimization, message passing neural network, deep reinforcement learning, multiple constraints

薛松萍, 高德荃, 赵子岩, 林彧茜, 广泽晶, 张大卫. 基于业务差异化传输需求下的电力通信网路由算法[J]. 中国电力, 2024, 57(11): 183-190.

Songping XUE, Dequan GAO, Ziyan ZHAO, Yuqian LIN, Zejing GUANG, Dawei ZHANG. Routing Algorithm for Power Communication Networks Based on Serivce Differentiated Transmission Requirements[J]. Electric Power, 2024, 57(11): 183-190.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202405020

https://www.electricpower.com.cn/CN/Y2024/V57/I11/183

图/表 21

图 1 福建省500 kV骨干网拓扑

Fig.1 Topology of 500 kV backbone network in Fujian Province

表 1 3类业务的归一化奖励

Table 1 Normalized reward for the three services

业务名称		奖励值
Ⅰ类生产控制业务		0.125
Ⅱ类生产控制业务		0.500
Ⅲ类生产控制业务		1.000

图 2 业务传输示例

Fig.2 Example of service transmission

表 2 不同业务的可靠性要求

Table 2 Reliability requirements for different services

业务名称		可靠性要求
Ⅰ类生产控制业务		0.90
Ⅱ类生产控制业务		0.88
Ⅲ类生产控制业务		0.86

表 3 不同业务的传输距离限制

Table 3 Transmission distance limitations for different services

业务名称		传输距离限制/km
Ⅰ类生产控制业务		1000
Ⅱ类生产控制业务		950
Ⅲ类生产控制业务		900

图 3 智能体与电力通信网交互

Fig.3 Agent interacting with the power communication network

图 4 消息传递神经网络的架构

Fig.4 Architecture of message passing neural network

图 5 算法1 消息传递神经网络

Fig.5 Algorithm 1 message passing neural network

图 6 深度强化学习算法的工作流程

Fig.6 Workflow of deep reinforcement learning algorithm

图 8 Loss曲线

Fig.8 Loss curve

表 4 训练时的超参数表

Table 4 Table of hyperparameters during training

超参数名称		超参数值
链路隐藏状态link_state_dim		32
学习率		0.0006
消息传递次数T		4
读出层层数readout_layers		3
读出层神经元个数readout_units		35
一次读出的数据个数batch_size		32
丢弃层丢弃概率dropout_rate		0.01

图 7 算法2 深度强化学习算法

Fig.7 Algorithm 2 Deep Reinforcement Learning Algorithm

图 9 得分曲线

Fig.9 Score curve

图 10 初始环境下的算法评估

Fig.10 Algorithm evaluation in the initial environment

图 11 Nsfnet拓扑

Fig.11 Nsfnet topology

图 12 Nsfnet下的算法评估

Fig.12 Algorithms evaluation under Nsfnet

图 13 随机拓扑

Fig.13 Random topology

图 14 随机拓扑下的算法评估

Fig.14 Algorithm evaluation under random topology

图 15 无故障下的算法评估

Fig.15 Algorithm evaluation without failures

图 16 1条链路故障下的算法评估

Fig.16 Algorithm evaluation under one link failure

图 17 2条链路故障下的算法评估

Fig.17 Algorithm evaluation under two link failures

参考文献 25

1	赵子岩, 刘建明. 基于业务风险均衡度的电力通信网可靠性评估算法[J]. 电网技术, 2011, 35 (10): 209- 213.
	ZHAO Ziyan, LIU Jianming. A new service risk balancing based method to evaluate reliability of electric power communication network[J]. Power System Technology, 2011, 35 (10): 209- 213.
2	夏超鹏. 泛在电力物联网在电力市场应用中的展望[J]. 发电技术, 2020, 41 (2): 142- 149. DOI
	XIA Chaopeng. Prospect of ubiquitous power Internet of Things in electricity market[J]. Power Generation Technology, 2020, 41 (2): 142- 149. DOI
3	王勇, 张明, 马洲俊, 等. 电力线信道通信特性影响因素分析[J]. 电力科学与技术学报, 2021, 36 (3): 157- 164, 173.
	WANG Yong, ZHANG Ming, MA Zhoujun, et al. Analysis of the factors influencing the communication characteristics of power line channel[J]. Journal of Electric Power Science and Technology, 2021, 36 (3): 157- 164, 173.
4	王志强, 吴庆, 张拯, 等. 基于异常分析的电力信息通信系统运维策略[J]. 陕西电力, 2016, 44 (4): 84- 87.
	WANG Zhiqiang, WU Qing, ZHANG Zheng, et al. Maintenance strategy of power grid information and telecommunication system based on abnormal analysis[J]. Shanxi Electric Power, 2016, 44 (4): 84- 87.
5	皮志勇, 朱益, 廖玄, 等. 基于深度学习的智能变电站通信链路故障定位方法[J]. 中国电力, 2023, 56 (7): 136- 145.
	PI Zhiyong, ZHU Yi, LIAO Xuan, et al. Fault location method for communication link in smart substation based on deep learning[J]. Electric Power, 2023, 56 (7): 136- 145.
6	皮志勇, 朱益, 廖玄, 等. 智能变电站多信息融合建模的通信链路故障定位方法[J]. 中国电力, 2023, 56 (8): 207- 215.
	PI Zhiyong, ZHU Yi, LIAO Xuan, et al. Fault location method for communication link with multi-information fusion modeling of smart substation[J]. Electric Power, 2023, 56 (8): 207- 215.
7	何天玲, 蔡立波. 耦合电网业务约束条件的电力通信故障检修辅助决策的实现[J/OL]. 中国电力, 2024: 1–8. (2024-03-21). https://kns.cnki.net/kcms/detail/11.3265.tm.20240320.1018.002.html.
	HE Tianling, CAI Libo. Implementation of power communication fault and maintenance aided decision-making coupled with power network service constraints[J/OL]. Electric Power, 2024: 1–8. (2024-03-21). https://kns.cnki.net/kcms/detail/11.3265.tm.20240320.1018.002.html.
8	李彬, 卢超, 景栋盛, 等. 负载与风险联合均衡的电力通信网路由优化算法[J]. 中国电机工程学报, 2019, 39 (9): 2713- 2722.
	LI Bin, LU Chao, JING Dongsheng, et al. An optimized routing algorithm with load and risk joint balance in electric communication network[J]. Proceedings of the CSEE, 2019, 39 (9): 2713- 2722.
9	崔力民, 孙静月, 李珊君, 等. 一种基于熵的电力通信网络业务资源均匀分配算法[J]. 电网技术, 2017, 41 (9): 3066- 3073.
	CUI Limin, SUN Jingyue, LI Shanjun, et al. An algorithm for business resource uniform distribution in power communication network based on entropy[J]. Power System Technology, 2017, 41 (9): 3066- 3073.
10	顿聪颖, 金凤林, 谭诗翰, 等. 基于负载均衡的低轨卫星网络动态路由算法[J]. 无线电工程, 2022, 52 (5): 790- 798. DOI
	DUN Congying, JIN Fenglin, TAN Shihan, et al. Dynamic routing algorithm for low orbit satellite network based on load balancing[J]. Radio Engineering, 2022, 52 (5): 790- 798. DOI
11	孙严智, 刘旋, 刘宇明, 等. 遗传算法电力通信网关键业务备选路由配置[J]. 云南电力技术, 2017, 45 (1): 113- 117. DOI
	SUN Yanzhi, LIU Xuan, LIU Yuming, et al. Research on key service backup routing method based on genetic algorithm in power communication network[J]. Yunnan Electric Power, 2017, 45 (1): 113- 117. DOI
12	王素红, 唐煜星, 郭文豪, 等. 考虑优先级的智能电网业务调度与资源分配方案[J]. 南方电网技术, 2024, 18 (4): 59- 70, 79.
	WANG Suhong, TANG Yuxing, GUO Wenhao, et al. Service scheduling and resource allocation scheme of smart grid considering priority[J]. Southern Power System Technology, 2024, 18 (4): 59- 70, 79.
13	刘思放. 电力需求响应业务传输机制及路由优化策略研究[D]. 北京: 华北电力大学, 2020.
	LIU Sifang. Research on transmission mechanism and routing optimized strategy for power demand response service[D]. Beijing: North China Electric Power University, 2020.
14	曾瑛, 李伟坚, 陈媛媛, 等. 基于业务优先级的电力调度数据网拥塞规避算法[J]. 电力系统保护与控制, 2014, 42 (2): 49- 55. DOI
	ZENG Ying, LI Weijian, CHEN Yuanyuan, et al. A congestion avoidance algorithm based on the service priority for electric power dispatching data network[J]. Power System Protection and Control, 2014, 42 (2): 49- 55. DOI
15	ALMASAN P, SUÁREZ-VARELA J, RUSEK K, et al. Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case[J]. Computer Communications, 2022, 196, 184- 194. DOI
16	YU C H, LAN J L, GUO Z H, et al. DROM: optimizing the routing in software-defined networks with deep reinforcement learning[J]. IEEE Access, 2018, 6, 64533- 64539. DOI
17	HARTERT R, VISSICCHIO S, SCHAUS P, et al. A declarative and expressive approach to control forwarding paths in carrier-grade networks[J]. ACM SIGCOMM Computer Communication Review, 2015, 45 (4): 15- 28. DOI
18	FRANCOIS P, FILSFILS C, EVANS J, et al. Achieving sub-second IGP convergence in large IP networks[J]. ACM SIGCOMM Computer Communication Review, 2005, 35 (3): 35- 44. DOI
19	JAIN S, KUMAR A, MANDAL S, et al. B4: Experience with a globally-deployed software defined WAN[J]. ACM SIGCOMM Computer Communication Review, 2013, 43 (4): 3- 14.
20	何天玲. 基于业务N-2的电力通信网可靠性评估方法研究[J]. 电力信息与通信技术, 2021, 19 (2): 36- 43.
	HE Tianling. Research on reliability evaluation method of power communication network based on service N-2[J]. Electric Power Information and Communication Technology, 2021, 19 (2): 36- 43.
21	王铮澄, 周艳真, 郭庆来, 等. 考虑电力系统拓扑变化的消息传递图神经网络暂态稳定评估[J]. 中国电机工程学报, 2021, 41 (7): 2341- 2349.
	WANG Zhengcheng, ZHOU Yanzhen, GUO Qinglai, et al. Transient stability assessment of power system considering topological change: a message passing neural network-based approach[J]. Proceedings of the CSEE, 2021, 41 (7): 2341- 2349.
22	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41 (1): 1- 27. DOI
	LIU Quan, ZHAI Jianwei, ZHANG Zongchang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41 (1): 1- 27. DOI
23	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning[J]. ArXiv e-Prints, 2013: arXiv: 1312.5602.
24	ABADI M, BARHAM P, CHEN J, et al. TensorFlow: a system for large-scale machine learning[C]//12th USENIX symposium on operating systems design and implementation (OSDI 16). 2016: 265–283.
25	HEI X J, ZHANG J, BENSAOU B, et al. Wavelength converter placement in least-load-routing-based optical networks using genetic algorithms[J]. Journal of Optical Networking, 2004, 3 (5): 363. DOI

[1]	曾仪, 周毅, 陆继翔, 周良才, 唐宁恺, 李红. 基于多智能体安全深度强化学习的电压控制[J]. 中国电力, 2025, 58(2): 111-117.
[2]	何天玲, 蔡立波. 耦合电网业务约束条件的电力通信故障检修辅助决策的实现[J]. 中国电力, 2024, 57(12): 178-187.
[3]	张超, 赵冬梅, 季宇, 张颖. 基于改进深度Q网络的虚拟电厂实时优化调度[J]. 中国电力, 2024, 57(1): 91-100.
[4]	石文喆, 李冰洁, 尤培培, 张泠. 基于深度强化学习的建筑能源系统优化策略[J]. 中国电力, 2023, 56(6): 114-122.
[5]	宋伟业, 刘灵玥, 阎洁, 王航宇, 何书凯, 韩爽, 王明辉, 刘永前. 基于深度强化学习的海上风电集群自进化功率平滑控制方法[J]. 中国电力, 2023, 56(3): 36-46.
[6]	张淑兴, 马驰, 杨志学, 王尧, 吴昊, 任洲洋. 基于深度确定性策略梯度算法的风光储系统联合调度策略[J]. 中国电力, 2023, 56(2): 68-76.
[7]	周庆锋, 王思淳, 李德鑫, 刘佳琪, 李同. 基于DDPG的风电场动态参数智能校核知识学习模型[J]. 中国电力, 2022, 55(5): 32-38.
[8]	杨朋朋, 王蓓蓓, 胥鹏, 王高琴, 郑亚先. 不完全信息下基于深度双Q网络的发电商三段式竞价策略[J]. 中国电力, 2021, 54(11): 47-58.
[9]	贾惠彬, 盖永贺, 李保罡, 郑宏达. 基于强化学习的电力通信网故障恢复方法[J]. 中国电力, 2020, 53(6): 34-40.
[10]	汪洋, 高晗星, 周生平, 王智慧, 何玉钧. 融合拓扑及业务特性的电力通信网关键节点识别[J]. 中国电力, 2018, 51(10): 111-118.
[11]	王玉东, 刘伟, 刘洁, 李威, 黄兴, 孟威. P-OTN技术在电力骨干通信网的应用及多业务承载策略研究[J]. 中国电力, 2017, 50(10): 28-34.
[12]	谢迎军, 王玉亭, 李炜, 丁慧霞, 王智慧, 张世先, 周阮坤. 电力企业通信网运行质量评估指标体系[J]. 中国电力, 2017, 50(10): 22-27.
[13]	曹原, 赵永利, 郁小松, 张杰. 量子密钥分发驱动安全电力通信网络体系架构[J]. 中国电力, 2017, 50(10): 8-11.
[14]	王一蓉，王艳茹，张荣博，武志栋. 基于SDN构建高性能、易扩展的电力通信网[J]. 中国电力, 2016, 49(10): 106-110.