Power Communication Network Recovery from Large-Scale Failures Based on Reinforcement Learning

doi:10.11930/j.issn.1004-9649.201907078

Abstract

Abstract: Natural disasters or malicious attacks may cause large-scale failures of power communication networks in smart grid systems, which will impose big risks on the security and stability of the power system operation unless the communication network is recovered immediately. In order to solve the recovery problem of power communication network after large-scale failure, under the constraints of limited link recovery resources, a link recovery model for large-scale failures in power communication networks is established with the objective to maximize the recovery amount of failed services. Regarding this model, a heuristic algorithm based on reinforcement learning is proposed, in which the link recovery resources and the degree of importance of the damaged link in the failed service are taken into account to set the reward and penalty functions as well as the selection rules. Then the optimal link recovery combination is obtained through the accumulation of the maximum reward values. The simulation results show that the power communication network failure recovery algorithm proposed in this paper can restore quite considerable failed services quickly with limited resources.

Key words: smart grid, power communication network, large-scale fault, reinforcement learning, fault recover

JIA Huibin, GAI Yonghe, LI Baogang, ZHENG Hongda. Power Communication Network Recovery from Large-Scale Failures Based on Reinforcement Learning[J]. Electric Power, 2020, 53(6): 34-40.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.electricpower.com.cn/EN/10.11930/j.issn.1004-9649.201907078

https://www.electricpower.com.cn/EN/Y2020/V53/I6/34

References

[1] 谢迎军, 王玉亭, 李炜, 等. 电力企业通信网运行质量评估指标体系[J]. 中国电力, 2017, 50(10): 22-27
XIE Yingjun, WANG Yuting, LI Wei, et al. Indicator system for operation quality of electric power communication network[J]. Electric Power, 2017, 50(10): 22-27
[2] CHIARADONNA S, DI GIANDOMENICO F, MASETTI G. Analyzing the impact of failures in the electric power distribution grid[C]//2016 Seventh Latin-American Symposium on Dependable Computing (LADC). Cali, Colombia. IEEE, 2016: 99-108.
[3] LI H T, LI Q, JIANG Y, et al. A declarative failure recovery system in software defined networks[C]//2016 IEEE International Conference on Communications (ICC). Kuala Lumpur, Malaysia. IEEE, 2016.
[4] BERNSTEIN A, BIENSTOCK D, HAY D, et al. Power grid vulnerability to geographically correlated failures: Analysis and control implications[C]//IEEE INFOCOM 2014 - IEEE Conference on Computer Communications. Toronto, ON, Canada. IEEE, 2014: 2634-2642.
[5] HASELTINE C, EMAN EMANE S. Prediction of power grid failure using neural network learning[C]//2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). Cancun, Mexico. IEEE, 2017: 505−510.
[6] FAN W L, LIU Z G, PING H, et al. Cascading failure model in power grids using the complex network theory[J]. IET Generation, Transmission & Distribution, 2016, 10(15): 3940-3949.
[7] 韩丽芳, 胡博文, 杨军, 等. 基于攻击预测的电力CPS安全风险评估[J]. 中国电力, 2019, 52(1): 48-56
HAN Lifang, HU Bowen, YANG Jun, et al. A new security risk assessment method for cyber physical power system based on attack prediction[J]. Electric Power, 2019, 52(1): 48-56
[8] 范君军, 陈智超, 陈涛, 等. 电网应急演练评估及其指标体系[J]. 中国电力, 2017, 50(11): 116-120
FAN Junjun, CHEN Zhichao, CHEN Tao, et al. Power grid emergency drill evaluation and the index system[J]. Electric Power, 2017, 50(11): 116-120
[9] 刁浩然, 杨明, 陈芳, 等. 基于强化学习理论的地区电网无功电压优化控制方法[J]. 电工技术学报, 2015, 30(12): 408-414
DIAO Haoran, YANG Ming, CHEN Fang, et al. Reactive power and voltage optimization control approach of the regional power grid based on reinforcement learning theory[J]. Transactions of China Electrotechnical Society, 2015, 30(12): 408-414
[10] MORADI M. A centralized reinforcement learning method for multi-agent job scheduling in Grid[C]//2016 6th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Iran. IEEE, 2016.
[11] YAN J, HE H B, ZHONG X N, et al. Q-learning-based vulnerability analysis of smart grid against sequential topology attacks[J]. IEEE Transactions on Information Forensics and Security, 2017, 12(1): 200-210.
[12] 汪洋, 高晗星, 周生平, 等. 融合拓扑及业务特性的电力通信网关键节点识别[J]. 中国电力, 2018, 51(10): 111-118
WANG Yang, GAO Hanxing, ZHOU Shengping, et al. Critical node identification for electric power communication network based on characteristics of topology and services[J]. Electric Power, 2018, 51(10): 111-118
[13] 陈国炎, 张哲, 尹项根, 等. 广域后备保护通信模式及其性能评估[J]. 中国电机工程学报, 2014, 34(1): 186-196
CHEN Guoyan, ZHANG Zhe, YIN Xianggen, et al. Wide area backup protection communication mode and its performance evaluation[J]. Proceedings of the CSEE, 2014, 34(1): 186-196
[14] 余才志. 必经节点的最短路径算法研究 [D]. 天津: 天津大学, 2016.
YU Caizhi. The research on the shortest path algorithm for the specified node[D]. Tianjin: Tianjin University, 2016.
[15] 康文雄, 许耀钊. 节点约束型最短路径的分层Dijkstra算法[J]. 华南理工大学学报(自然科学版), 2017, 45(1): 66-73
KANG Wenxiong, XU Yaozhao. A hierarchical dijkstra algorithm for solving shortest path from constrained nodes[J]. Journal of South China University of Technology(Natural Science Edition), 2017, 45(1): 66-73
[16] 杨春霞. 网络大规模毁坏后的渐进恢复机制研究[D]. 成都: 电子科技大学, 2012.
[17] PILEROOD A E, HEYDARI M, MAZDEH M M. A two-stage greedy heuristic for a flowshop scheduling problem under time-of-use electricity tariffs[J]. South African Journal of Industrial Engineering, 2018,29(1):143-154.
[18] 何瑞江, 胡志坚, 李燕, 等. 含分布式电源配电网故障区段定位的线性整数规划方法[J]. 电网技术, 2018, 42(11): 3684-3692
HE Ruijiang, HU Zhijian, LI Yan, et al. Fault section location method for DG-DNs based on integer linear programming[J]. Power System Technology, 2018, 42(11): 3684-3692
[19] 崔文岩, 孟相如, 杨欢欢, 等. QoS约束的链路故障多备份路径恢复算法[J]. 电子与信息学报, 2016, 38(8): 1850-1857
CUI Wenyan, MENG Xiangru, YANG Huanhuan, et al. Link failure recovery algorithm based on multiple backup paths with QoS constraint[J]. Journal of Electronics & Information Technology, 2016, 38(8): 1850-1857
[20] 陈晓玲, 杨军, 罗超, 等. 一种大电网潮流转移路径快速搜索方法[J]. 电网技术, 2015, 39(4): 1045-1052
CHEN Xiaoling, YANG Jun, LUO Chao, et al. A high-speed searching method for power flow transferring paths in large power grid[J]. Power System Technology, 2015, 39(4): 1045-1052