中国电力 ›› 2025, Vol. 58 ›› Issue (10): 50-62.DOI: 10.11930/j.issn.1004-9649.202503064
• “十五五”电力系统源网荷储协同规划运行关键技术 • 上一篇 下一篇
柳华1(
), 熊再豹1(
), 蒋陶宁1(
), 高宇1(
), 金雨含2(
), 葛磊蛟2(
)
收稿日期:2025-03-20
发布日期:2025-10-23
出版日期:2025-10-28
作者简介:基金资助:
LIU Hua1(
), XIONG Zaibao1(
), JIANG Taoning1(
), GAO Yu1(
), JIN Yuhan2(
), GE Leijiao2(
)
Received:2025-03-20
Online:2025-10-23
Published:2025-10-28
Supported by:摘要:
随着全球能源需求增长和可持续发展目标推进,微电网能量管理面临高维度、复杂性和动态性挑战。因此,提出一种分布式强化学习驱动的微电网群能量优化管理策略,旨在通过智能化手段提升微电网群在能源调度和管理方面的效率。首先,针对微电网群负荷动态变化大、拓扑结构复杂等难题,构建目标优化函数,并引入一种分布式强化学习算法,实现微电网群在分布式环境下的自适应决策与协同优化。其次,将微电网中的每个电源点视为一个智能体,利用信息共享实现全局效益最大化与发电成本最小化,达到微电网群发电、储能和负荷需求管理的实时优化。最后,通过实际案例进行验证,结果表明所提策略能够维持电力供需之间的动态平衡,与传统方法技术相比,总发电成本节约了18%左右。
柳华, 熊再豹, 蒋陶宁, 高宇, 金雨含, 葛磊蛟. 分布式强化学习驱动的微电网群动态能量优化管理策略[J]. 中国电力, 2025, 58(10): 50-62.
LIU Hua, XIONG Zaibao, JIANG Taoning, GAO Yu, JIN Yuhan, GE Leijiao. Distributed Reinforcement Learning-Driven Dynamic Energy Optimization Management Strategy for Microgrid Clusters[J]. Electric Power, 2025, 58(10): 50-62.
| 预测器 | 验证集均方误差 | 测试集均方误差 | ||
| Linear model | 0.080 | 0.078 | ||
| Dense | 0.065 | 0.061 | ||
| Multi-step dense | 0.063 | 0.060 | ||
| CNN | 0.055 | 0.056 | ||
| LSTM | 0.048 | 0.050 |
表 1 预测性能
Table 1 Predicting performance
| 预测器 | 验证集均方误差 | 测试集均方误差 | ||
| Linear model | 0.080 | 0.078 | ||
| Dense | 0.065 | 0.061 | ||
| Multi-step dense | 0.063 | 0.060 | ||
| CNN | 0.055 | 0.056 | ||
| LSTM | 0.048 | 0.050 |
| 发电机 | Pmin/kW | Pmax/kW | a | b | c | |||||
| 1 | 200 | 500 | 10 | |||||||
| 2 | 100 | 300 | 8 | |||||||
| 3 | 200 | 100 | 6 |
表 2 发电机参数
Table 2 Generator parameters
| 发电机 | Pmin/kW | Pmax/kW | a | b | c | |||||
| 1 | 200 | 500 | 10 | |||||||
| 2 | 100 | 300 | 8 | |||||||
| 3 | 200 | 100 | 6 |
| 方法 | 单次迭代时间/ms | 收敛迭代次数 | 总计算时间/s | |||
| 传统方法 | 125 | 48 | 6.00 | |||
| MS-MADDPG | 210 | 18 | 3.78 |
表 3 计算效率对比
Table 3 Computational efficiency comparison
| 方法 | 单次迭代时间/ms | 收敛迭代次数 | 总计算时间/s | |||
| 传统方法 | 125 | 48 | 6.00 | |||
| MS-MADDPG | 210 | 18 | 3.78 |
| 场景 | 占比/% | |||||
| 光伏 | 风电 | 柴油机 | ||||
| 低渗透率 | 20 | 10 | 90 | |||
| 中渗透率 | 40 | 30 | 30 | |||
| 高渗透率 | 60 | 30 | 10 | |||
表 4 可再生能源渗透率占比
Table 4 Percentage of renewable energy penetration
| 场景 | 占比/% | |||||
| 光伏 | 风电 | 柴油机 | ||||
| 低渗透率 | 20 | 10 | 90 | |||
| 中渗透率 | 40 | 30 | 30 | |||
| 高渗透率 | 60 | 30 | 10 | |||
| 方法 | 学习结构 | Mean diff | Std | 奖励 | ||||
| DDPG | 集中式 | – | ||||||
| MS-MADDPG | 分布式 | – | ||||||
| 在线ADMM | 分布式 | – | ||||||
| 传统方法 | 分布式 | – |
表 5 性能比较
Table 5 Performance comparison
| 方法 | 学习结构 | Mean diff | Std | 奖励 | ||||
| DDPG | 集中式 | – | ||||||
| MS-MADDPG | 分布式 | – | ||||||
| 在线ADMM | 分布式 | – | ||||||
| 传统方法 | 分布式 | – |
| 1 | 王治国, 刘继荣, 郝艳军, 等. 多智能体系统微电网能源实时管理系统设计[J]. 信息技术, 2021, 45 (5): 61- 67. |
| WANG Zhiguo, LIU Jirong, HAO Yanjun, et al. Design of real time energy management system for multi intelligent microgrid[J]. Information Technology, 2021, 45 (5): 61- 67. | |
| 2 |
魏震波, 张芷琪, 李银江, 等. 多主体合作模式下微电网规划运行一体化模型[J]. 电力建设, 2024, 45 (10): 47- 58.
DOI |
|
WEI Zhenbo, ZHANG Zhiqi, LI Yinjiang, et al. An integrated model of microgrid planning and operation under A multi-subject cooperation model[J]. Electric Power Construction, 2024, 45 (10): 47- 58.
DOI |
|
| 3 |
FARZIN H, FOTUHI-FIRUZABAD M, MOEINI-AGHTAIE M. A stochastic multi-objective framework for optimal scheduling of energy storage systems in microgrids[J]. IEEE Transactions on Smart Grid, 2017, 8 (1): 117- 127.
DOI |
| 4 | 樊晓伟, 王瑞妙, 杨海峰, 等. 计及源荷不确定的综合能源微电网集群优化运行[J]. 电力建设, 2024, 45 (8): 128- 139. |
| FAN Xiaowei, WANG Ruimiao, YANG Haifeng, et al. Optimization operation of integrated energy microgrid cluster considering source-load uncertainty[J]. Electric Power Construction, 2024, 45 (8): 128- 139. | |
| 5 | 邱革非, 冯泽华, 沈赋, 等. 考虑车网互动的园区电网动态双层能量管理策略[J]. 上海交通大学学报, 2024, 58 (6): 916- 925. |
| QIU Gefei, FENG Zehua, SHEN Fu, et al. Dynamic double-layer energy management strategy for park power grid considering vehicle-to-grid[J]. Journal of Shanghai Jiao Tong University, 2024, 58 (6): 916- 925. | |
| 6 |
ZHENG Z, SHAFIQUE M, LUO X W, et al. A systematic review towards integrative energy management of smart grids and urban energy systems[J]. Renewable and Sustainable Energy Reviews, 2024, 189, 114023.
DOI |
| 7 |
徐明宇, 郝文波, 王盼宝, 等. 基于动态随机模型的微电网群能量管理方法[J]. 电力工程技术, 2022, 41 (5): 140- 148.
DOI |
|
XU Mingyu, HAO Wenbo, WANG Panbao, et al. Energy management method of multi-microgrids based on dynamic stochastic model[J]. Electric Power Engineering Technology, 2022, 41 (5): 140- 148.
DOI |
|
| 8 | 李扬, 马文捷, 卜凡金, 等. 多智能体深度强化学习驱动的跨园区能源交互优化调度[J]. 电力建设, 2024, 45 (5): 59- 70. |
| LI Yang, MA Wenjie, BU Fanjin, et al. Deep reinforcement learning-driven cross-community energy interaction optimal scheduling[J]. Electric Power Construction, 2024, 45 (5): 59- 70. | |
| 9 |
EDUSSURIYA C, MARIKKAR U, WICKRAMASINGHE S, et al. Peer-to-peer energy trading through swarm intelligent stackelberg game[J]. Energies, 2023, 16 (5): 2434.
DOI |
| 10 | 张考, 何凯琳, 杨沛豪. 基于模糊强化学习的电力变压器故障诊断算法研究[J]. 综合智慧能源, 2024, 46 (10): 48- 55. |
| ZHANG Kao, HE Kailin, YANG Peihao. Research on power transformer fault diagnosis algorithm based on fuzzy reinforcement learning[J]. Integrated Intelligent Energy, 2024, 46 (10): 48- 55. | |
| 11 | 党亚峥, 唐崇伟. 一种改进的乘子交替方向法在ℓ1-正则化分裂可行问题中的应用[J]. 上海理工大学学报, 2020, 42 (5): 460- 466,503. |
| DANG Yazheng, TANG Chongwei. An improved alternating direction method of multipliers for ℓ1-norm regularization splitting feasibility problem[J]. Journal of University of Shanghai for Science and Technology, 2020, 42 (5): 460- 466,503. | |
| 12 |
YIASEMIS G, MORIAKOV N, SONKE J J, et al. vSHARP: variable splitting half-quadratic ADMM algorithm for reconstruction of inverse-problems[J]. Magnetic Resonance Imaging, 2025, 115, 110266.
DOI |
| 13 |
VARMA A V S S, MANEPALLI K. VLSI realization of hybrid fast fourier transform using reconfigurable booth multiplier[J]. International Journal of Information Technology, 2024, 16 (7): 4323- 4333.
DOI |
| 14 | 苏一丹, 续嘉, 覃华. 二元裂解算子交替方向乘子法的核极限学习机[J]. 电子与信息学报, 2021, 43 (9): 2586- 2593. |
| SU Yidan, XU Jia, QIN Hua. Kernel extreme learning machine based on alternating direction multiplier method of binary splitting operator[J]. Journal of Electronics & Information Technology, 2021, 43 (9): 2586- 2593. | |
| 15 | HASSAN IBRAHIM S, SALAH HAMEED I, HADI ALI H. High performance IIR filter design based on fast multiplier[J]. Diyala Journal of Engineering Sciences, 2025,192–202. |
| 16 | 刘洋, 薛中会, 王永全, 等. 分裂可行性问题的外推加速线性交替方向乘子法及其全局收敛性[J]. 计算机科学, 2023, 50 (6): 261- 265. |
| LIU Yang, XUE Zhonghui, WANG Yongquan, et al. Extrapolation accelerated linear alternating direction multiplier method for split feasibility problems and its global convergence[J]. Computer Science, 2023, 50 (6): 261- 265. | |
| 17 | GAJAWADA S, DEVI D N, RAO M. MOHSKM: meta-heuristic optimization driven hardware-efficient heterogeneous-split karatsuba multipliers for large-bit operations[C]//2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). Knoxville, TN, USA. IEEE, 2024: 749-752. |
| 18 |
LEI W L, YE Y, XIAO M, et al. Adaptive stochastic ADMM for decentralized reinforcement learning in edge IoT[J]. IEEE Internet of Things Journal, 2022, 9 (22): 22958- 22971.
DOI |
| 19 |
ANNESE M, FERNÁNDEZ M A, GASTALDI L. Splitting schemes for a Lagrange multiplier formulation of FSI with immersed thin-walled structure: stability and convergence analysis[J]. IMA Journal of Numerical Analysis, 2023, 43 (2): 881- 919.
DOI |
| 20 |
ARWA E O, FOLLY K A. Reinforcement learning techniques for optimal power control in grid-connected microgrids: a comprehensive review[J]. IEEE Access, 2020, 8, 208992- 209007.
DOI |
| 21 |
ZHU D, YANG B, LIU Y, et al. Energy management based on multi-agent deep reinforcement learning for a multi-energy industrial park[J]. Applied Energy, 2022, 311, 118636.
DOI |
| 22 |
MATTERA G, CAGGIANO A, NELE L. Optimal data-driven control of manufacturing processes using reinforcement learning: an application to wire arc additive manufacturing[J]. Journal of Intelligent Manufacturing, 2025, 36 (2): 1291- 1310.
DOI |
| 23 |
DAI P C, YU W W, WEN G H, et al. Distributed reinforcement learning algorithm for dynamic economic dispatch with unknown generation cost functions[J]. IEEE Transactions on Industrial Informatics, 2020, 16 (4): 2258- 2267.
DOI |
| 24 |
XIAO G Y, ZHANG H G. Convergence analysis of value iteration adaptive dynamic programming for continuous-time nonlinear systems[J]. IEEE Transactions on Cybernetics, 2024, 54 (3): 1639- 1649.
DOI |
| 25 |
DESHPANDE S V, HARIKRISHNAN R, IBRAHIM B S K K, et al. Mobile robot path planning using deep deterministic policy gradient with differential gaming (DDPG-DG) exploration[J]. Cognitive Robotics, 2024, 4, 156- 173.
DOI |
| 26 | 孙国强, 殷岩岩, 卫志农, 等. 基于深度确定性策略梯度的主动配电网有功-无功协调优化调度[J]. 电力建设, 2023, 44 (11): 33- 42. |
| SUN Guoqiang, YIN Yanyan, WEI Zhinong, et al. Coordinated optimal dispatch of active and reactive power in active distribution networks using deep deterministic strategy gradient[J]. Electric Power Construction, 2023, 44 (11): 33- 42. | |
| 27 | 黄堃, 付明, 梁加本. 基于融合专家知识DDPG的孤岛微电网频率调节策略[J]. 中国电力, 2024, 57 (2): 194- 201. |
| HUANG Kun, FU Ming, LIANG Jiaben. Frequency regulation strategy of isolated island microgrid based on fusion expert knowledge DDPG[J]. Electric Power, 2024, 57 (2): 194- 201. | |
| 28 |
SHI X T, LI Y J, HU W X, et al. Optimal lateral path-tracking control of vehicles with partial unknown dynamics via DPG-based reinforcement learning methods[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9 (1): 1701- 1710.
DOI |
| 29 |
CARTA S, FERREIRA A, PODDA A S, et al. Multi-DQN: an ensemble of deep Q-learning agents for stock market forecasting[J]. Expert Systems with Applications, 2021, 164, 113820.
DOI |
| 30 | 吴润泽, 霍金鑫, 郭昊博. 基于DQN的电力协同计算与缓存的任务卸载策略[J]. 电力建设, 2024, 45 (8): 149- 158. |
| WU Runze, HUO Jinxin, GUO Haobo. DQN-based task offloading strategy for power co-computing and caching[J]. Electric Power Construction, 2024, 45 (8): 149- 158. | |
| 31 | 万玲玲, 陈中, 王毅, 等. 考虑能量时空转移的城市规模化共享电动汽车充放电优化调度[J]. 电力建设, 2023, 44 (6): 135- 143. |
| WAN Lingling, CHEN Zhong, WANG Yi, et al. Optimal charging and discharging scheduling of urban large-scale shared electric vehicles considering energy temporal and spatial transfer[J]. Electric Power Construction, 2023, 44 (6): 135- 143. | |
| 32 |
YE Y, CHEN H, XIAO M, et al. Privacy-preserving incremental ADMM for decentralized consensus optimization[J]. IEEE Transactions on Signal Processing, 2020, 68, 5842- 5854.
DOI |
| [1] | 樊会丛, 段志国, 陈志永, 朱士加, 刘航, 李文霄, 杨阳. 基于多智能体深度策略梯度的离网型微电网双层优化调度[J]. 中国电力, 2025, 58(5): 11-20, 32. |
| [2] | 周飞航, 王灏, 王海利, 王萌, 金耀杰, 李重春, 张忠德, 王鹏. 基于多智能体强化学习的电-碳-绿证耦合市场下多市场主体行为研究[J]. 中国电力, 2025, 58(4): 44-55. |
| [3] | 曾仪, 周毅, 陆继翔, 周良才, 唐宁恺, 李红. 基于多智能体安全深度强化学习的电压控制[J]. 中国电力, 2025, 58(2): 111-117. |
| [4] | 何锦涛, 王灿, 王明超, 程本涛, 刘于正, 常文涵, 王锐, 余涵. 基于改进双深度Q网络的微电网群能量管理策略[J]. 中国电力, 2025, 58(10): 14-26. |
| [5] | 李超英, 檀勤良. 基于智能体建模的新型电力系统下火电企业市场交易策略[J]. 中国电力, 2024, 57(2): 212-225. |
| [6] | 张兴平, 王腾, 张馨月, 张浩楠. 基于多智能体深度确定策略梯度算法的火力发电商竞价策略[J]. 中国电力, 2024, 57(11): 161-172. |
| [7] | 丁雨, 于艾清, 高纯. 基于改进一致性算法的独立光储直流微电网电压稳定能量协调策略[J]. 中国电力, 2022, 55(3): 74-79. |
| [8] | 李咸善, 陈奥博, 程杉, 陈敏睿. 基于生态博弈的含云储能微电网多智能体协调优化调度[J]. 中国电力, 2021, 54(7): 166-177. |
| [9] | 赵凤贤,吴静,孙丽颖. 基于MAPSO优化的智能配电网大面积断电供电恢复[J]. 中国电力, 2016, 49(1): 85-90. |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||


AI小编