基于对抗性强化学习的配电网融合通信效率提升方法

doi:10.11930/j.issn.1004-9649.202210068

中国电力 ›› 2023, Vol. 56 ›› Issue (9): 127-133.DOI: 10.11930/j.issn.1004-9649.202210068

• 配电网规划与优化运行 • 上一篇下一篇

基于对抗性强化学习的配电网融合通信效率提升方法

彭琳钰¹, 刘旭¹, 汤玮¹, 刘晴¹, 方浩², 张光辉³

1. 贵州电网有限责任公司，贵州贵阳 550002;
2. 贵州电网有限责任公司贵阳供电局，贵州贵阳 550004;
3. 贵州电网有限责任公司电力调度控制中心，贵州贵阳 550002

收稿日期:2022-10-18 修回日期:2023-08-10 发布日期:2023-09-20
作者简介:彭琳钰(1993-),女,硕士,助理工程师,从事电力系统通信研究,E-mail:penglinyu666@163.com;刘旭(1984-),男,通信作者,硕士,高级工程师,从事电力系统通信研究,E-mail:290698192@qq.com;汤玮(1988-),男,硕士,高级工程师,从事电力系统通信研究,E-mail:tangwei@gz.csg.cn;刘晴(1976-),女,硕士,高级工程师,从事电力系统通信研究,E-mail:liuqing1949@sina.com;方浩(1976-),男,工程师,从事配网监控研究,E-mail:fanghao@gz.csg.cn;张光辉(1995-),男,助理工程师,从事电力系统通信研究,E-mail:zgh199509@163.com
基金资助:
中国南方电网有限责任公司科技项目（066500GS62200017）。

Adversarial Reinforcement Learning-Based Converged Communication Efficiency Improvement Method for Power Distribution Network

PENG Linyu¹, LIU Xu¹, TANG Wei¹, LIU Qing¹, FANG Hao², ZHANG Guanghui³

1. Guizhou Power Grid Co., Ltd., Guiyang 550002, China;
2. Guiyang Power Supply Bureau of Guizhou Power Grid Co., Ltd., Guiyang 550004, China;
3. Guizhou Power Dispatching & Communication Center, Guiyang 550002, China

Received:2022-10-18 Revised:2023-08-10 Published:2023-09-20
Supported by:
This work is supported by Science and Technology Project of China Southern Power Grid Corporation (No.066500GS62200017).

摘要/Abstract

摘要： 为了满足海量配电网终端源节点的多样化通信需求，须对配电网融合通信进行合理的编排优化。基于此，首先构建数据传输时延与传输能耗联合优化问题；其次，将优化问题建模为多臂赌博机问题，并提出一种基于对抗性强化学习的配电网融合通信编排算法，利用历史编排信息和感知源节点间的对抗性，动态学习通信编排决策；最后，通过仿真验证所提算法的优越性能。

关键词: 配电网, 强化学习, 对抗性感知, 通信编排

Abstract: In order to satisfy the diversified communication requirements of terminal source nodes in power distribution network, it is necessary to optimize the communication orchestration in power distribution unified communication network. Firstly, we construct the joint optimization problem of data transmission delay and energy consumption. Then, the joint optimization problem is modeled as a multi-armed bandit problem, and an adversarial reinforcement learning-based communication orchestration algorithm for power distribution unified communication network is proposed, which uses the historical orchestration information and the perceived adversary between source nodes to dynamically learn the communication orchestration strategy. Finally, the superior performance of the proposed algorithm is verified through simulation.

Key words: distribution network, reinforcement learning, adversary awareness, communication orchestration

彭琳钰, 刘旭, 汤玮, 刘晴, 方浩, 张光辉. 基于对抗性强化学习的配电网融合通信效率提升方法[J]. 中国电力, 2023, 56(9): 127-133.

PENG Linyu, LIU Xu, TANG Wei, LIU Qing, FANG Hao, ZHANG Guanghui. Adversarial Reinforcement Learning-Based Converged Communication Efficiency Improvement Method for Power Distribution Network[J]. Electric Power, 2023, 56(9): 127-133.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202210068

https://www.electricpower.com.cn/CN/Y2023/V56/I9/127

参考文献

[1] 王成山, 李鹏. 分布式发电、微网与智能配电网的发展与挑战[J]. 电力系统自动化, 2010, 34(2): 10–14, 23
WANG Chengshan, LI Peng. Development and challenges of distributed generation, the micro-grid and smart distribution system[J]. Automation of Electric Power Systems, 2010, 34(2): 10–14, 23
[2] 蔡乾, 王晶, 耿天翔, 等. 考虑新能源资源及出力特性的全局备用容量优化方法[J]. 中国电力, 2021, 54(2): 90–97
CAI Qian, WANG Jing, GENG Tianxiang, et al. Global reserve optimization method considering resources and output characteristics of renewable energy[J]. Electric Power, 2021, 54(2): 90–97
[3] 方如举, 葛瑜, 孙伟, 等. 基于WSNs的智能配电网通信数据传输带宽的优化分配策略[J]. 电力系统保护与控制, 2021, 49(23): 88–95
FANG Ruju, GE Yu, SUN Wei, et al. Transmission bandwidth optimal allocation strategy of communication data for a smart distribution grid based on WSNs[J]. Power System Protection and Control, 2021, 49(23): 88–95
[4] 李沛哲, 肖振锋, 陈仲伟, 等. 电力终端通信接入网通信技术匹配[J]. 电力科学与技术学报, 2021, 36(3): 125–134
LI Peizhe, XIAO Zhenfeng, CHEN Zhongwei, et al. Analysis of communication matching technology of power terminal communication access network[J]. Journal of Electric Power Science and Technology, 2021, 36(3): 125–134
[5] 杨振铨, 项基, 李艳君. 配合主网调度的配电网分布式电源主动控制策略[J]. 中国电机工程学报, 2019, 39(11): 3176–3186
YANG Zhenquan, XIANG Ji, LI Yanjun. Active control strategy of distributed generations for utility grid cooperation in distribution network[J]. Proceedings of the CSEE, 2019, 39(11): 3176–3186
[6] LIU J M, ZHAO Z Y, JI J, et al. Research and application of wireless sensor network technology in power transmission and distribution system[J]. Intelligent and Converged Networks, 2020, 1(2): 199–220.
[7] TANG Y J, CHENG N, WU W, et al. Delay-minimization routing for heterogeneous VANETs with machine learning based mobility prediction[J]. IEEE Transactions on Vehicular Technology, 2019, 68(4): 3967–3979.
[8] APRUZZESE G, ANDREOLINI M, MARCHETTI M, et al. Deep reinforcement adversarial learning against botnet evasion attacks[J]. IEEE Transactions on Network and Service Management, 2020, 17(4): 1975–1987.
[9] 吕昊, 何益鸣, 田浩, 等. 基于物联网的园区综合能源系统快速通信网络建模与仿真[J]. 中国电力, 2022, 55(5): 166–173
LV Hao, HE Yiming, TIAN Hao, et al. Modeling and simulation of fast communication network for park integrated energy system based on IoT[J]. Electric Power, 2022, 55(5): 166–173
[10] LIAO H J, ZHOU Z Y, MUMTAZ S, et al. Robust task offloading for IoT fog computing under information asymmetry and information uncertainty[C]//ICC 2019 - 2019 IEEE International Conference on Communications (ICC). Shanghai, China. IEEE, 2019: 1–6.
[11] 陶维青, 窦开明, 方陈, 等. 配电网相量数据接入方式比较及技术分析[J]. 电网技术, 2019, 43(3): 784–792
TAO Weiqing, DOU Kaiming, FANG Chen, et al. Comparison and technical analysis of phasor data access modes in distribution network[J]. Power System Technology, 2019, 43(3): 784–792
[12] 王玉东, 刘伟, 刘洁, 等. P-OTN技术在电力骨干通信网的应用及多业务承载策略研究[J]. 中国电力, 2017, 50(10): 28–34
WANG Yudong, LIU Wei, LIU Jie, et al. Research on application of P-OTN technology in electric power backbone communication network and multi-service bearing strategy[J]. Electric Power, 2017, 50(10): 28–34
[13] 梁云, 黄莉, 侯兴哲, 等. 配电网CPS的通信需求和网络资源分配方案探讨[J]. 中国电力, 2019, 52(1): 32–39
LIANG Yun, HUANG Li, HOU Xingzhe, et al. Discussion on communication requirement and network resource allocation scheme of cyber physical system for distribution network[J]. Electric Power, 2019, 52(1): 32–39
[14] BANAGAR M, MAHAM B, POPOVSKI P, et al. Power distribution of device-to-device communications in underlaid cellular networks[J]. IEEE Wireless Communications Letters, 2016, 5(2): 204–207.
[15] 王建平, 王梦彪, 王金玲, 等. 智能配电网通信实时性与可靠性QoS路由机制研究[J]. 电子测量与仪器学报, 2013, 27(3): 187–193
WANG Jianping, WANG Mengbiao, WANG Jinling, et al. Research on real-time and reliability of intelligent distribution networks QoS routing mechanism[J]. Journal of Electronic Measurement and Instrument, 2013, 27(3): 187–193
[16] SUN W, YUAN X J, WANG J P, et al. Quality of service networking for smart grid distribution monitoring[C]//2010 First IEEE International Conference on Smart Grid Communications. Gaithersburg, MD, USA. IEEE, 2010: 373–378.
[17] MOUSTAPHA A I, SELMIC R R. Wireless sensor network modeling using modified recurrent neural networks: application to fault detection[J]. IEEE Transactions on Instrumentation and Measurement, 2008, 57(5): 981–988.
[18] DHURANDHER S K, MISRA S, OBAIDAT M S, et al. An ant colony optimization approach for reputation and quality-of-service-based security in wireless sensor networks[J]. Security and Communication Networks, 2009, 2(2): 215–224.
[19] ZHOU Z Y, CHEN X Y, LIAO H J, et al. Context-aware learning-based resource allocation for ubiquitous power IoT[J]. IEEE Internet of Things Magazine, 2020, 3(4): 46–52.
[20] 王毅, 王智微, 何新. 智能电站数据中台建设与应用[J]. 中国电力, 2021, 54(3): 61–67, 176
WANG Yi, WANG Zhiwei, HE Xin. Data middle platform construction and application of intelligent power stations[J]. Electric Power, 2021, 54(3): 61–67, 176
[21] HONG A, LUNSCHER N, HU T H, et al. A multimodal emotional human–robot interaction architecture for social robots engaged in bidirectional communication[J]. IEEE Transactions on Cybernetics, 2021, 51(12): 5954–5968.
[22] ZHANG S X, LIAO H J, ZHOU Z Y, et al. Federated deep actor-critic-based task offloading in air-ground electricity IoT[C]//2021 IEEE Global Communications Conference (GLOBECOM). Madrid, Spain. IEEE, 2022: 1–6.
[23] WANG Z, JIA Z H, LIAO H J, et al. Energy-aware and URLLC-aware task offloading for Internet of health things[C]//GLOBECOM 2020-2020 IEEE Global Communications Conference. Taipei, China. IEEE, 2021: 1–6.
[24] JIA Z H, LIAO H J, ZHOU Z Y, et al. Multi-dimension resource allocation for NOMA-edge computing-based 6G power IoT[C]//2021 IEEE International Conference on Communications Workshops (ICC Workshops). Montreal, QC, Canada. IEEE, 2021: 1–6.
[25] LIAO H J, ZHOU Z Y, ZHAO X W, et al. Task offloading for vehicular fog computing under information uncertainty: a matching-learning approach[C]//2019 15 th International Wireless Communications & Mobile Computing Conference (IWCMC). Tangier, Morocco. IEEE, 2019: 2001–2006.

基于对抗性强化学习的配电网融合通信效率提升方法

Adversarial Reinforcement Learning-Based Converged Communication Efficiency Improvement Method for Power Distribution Network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	曾仪, 周毅, 陆继翔, 周良才, 唐宁恺, 李红. 基于多智能体安全深度强化学习的电压控制[J]. 中国电力, 2025, 58(2): 111-117.
[2]	贺春光, 王林峰, 曹媛, 安佳坤, 雷子健, 宋关羽, 冀浩然. 考虑综合收益的多电压等级配电网柔性互联装置协同规划方法[J]. 中国电力, 2025, 58(1): 78-84.
[3]	祝士焱, 许寅, 和敬涵, 王颖. 基于多微电网投影的配电系统协调恢复方法[J]. 中国电力, 2024, 57(9): 224-230.
[4]	张亚健, 陈茨, 薛飞, 马丽, 郑敏. 电制氢协同的含高比例光伏配电网两阶段电压随机优化控制[J]. 中国电力, 2024, 57(8): 23-35.
[5]	孙东磊, 孙毅, 刘蕊, 孙鹏凯, 张玉敏. 计及多层级配电网的分布式新能源最大消纳空间分解测算[J]. 中国电力, 2024, 57(8): 108-116.
[6]	凡鹏飞, 李宝琴, 侯江伟, 李嵘, 宋崇明, 林凯骏. 配电网分布式电源经济可承载力评估[J]. 中国电力, 2024, 57(7): 196-202.
[7]	王家武, 赵佃云, 刘长锋, 陈康, 张玉敏. 基于目标级联法的多主体主动配电网自治协同优化[J]. 中国电力, 2024, 57(7): 214-226.
[8]	韩璟琳, 胡平, 侯若松, 陈志永, 李洪涛, 柴园园. 计及多光储一体机的配电网电压优化控制策略[J]. 中国电力, 2024, 57(6): 69-77, 152.
[9]	徐峰亮, 王克谦, 王文豪, 王鹏, 王文烨, 张帅, 赵凤展. 计及激励型需求响应的低压配电网混合储能优化配置[J]. 中国电力, 2024, 57(6): 90-101.
[10]	乔俊峰, 周爱华, 彭林, 王一清, 沈晓峰, 潘森, 杨佩, 黄晨宏. 基于多源数据深度融合的配电网运行评价方法[J]. 中国电力, 2024, 57(6): 193-203.
[11]	张宇, 魏新迟, 冯凯辉, 时珊珊, 孟子涵, 梁媛. 计及分布式光伏承载力的配电网侧独立储能充放电策略[J]. 中国电力, 2024, 57(4): 111-117.
[12]	魏明江, 李鹏, 于浩, 冀浩然, 宋关羽, 习伟. 数字配电网边缘计算模拟实验平台[J]. 中国电力, 2024, 57(3): 12-19.
[13]	焦昊, 殷岩岩, 吴晨, 刘建, 徐春雷, 徐贤, 孙国强. 基于安全强化学习的主动配电网有功-无功协调优化调度[J]. 中国电力, 2024, 57(3): 43-50.
[14]	刘国强, 李国春, 郝亚楠, 李贵海, 吴中杰. 低压直流配电网强弱电弧特征及检测算法[J]. 中国电力, 2024, 57(2): 94-102.
[15]	李超英, 檀勤良. 基于智能体建模的新型电力系统下火电企业市场交易策略[J]. 中国电力, 2024, 57(2): 212-225.

模态框（Modal）标题

基于对抗性强化学习的配电网融合通信效率提升方法

Adversarial Reinforcement Learning-Based Converged Communication Efficiency Improvement Method for Power Distribution Network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics