基于融合专家知识DDPG的孤岛微电网频率调节策略

doi:10.11930/j.issn.1004-9649.202302031

摘要/Abstract

摘要：

随着风、光等间歇性新能源接入到孤岛微电网，传统控制方法在进行频率调节时难以有效协同源-荷-储等多种资源以应对源-荷的随机性波动所导致的频率偏差问题。为此，提出了一种融合专家知识与深度确定性策略梯度（DDPG）的孤岛微电网频率调节算法，通过专家知识的经验规则引导各调控设备与环境高效交互，提升多资源协同频率调节的性能。仿真结果表明所提调频策略能够充分挖掘微网内多种资源的调频潜力，并有效提升调频性能。

关键词: 孤岛微电网, 频率调节, 专家知识, 深度确定性策略梯度

Abstract:

With the wide access of intermittent renewable energy such as wind and PV to island microgrid, it is difficult for traditional control methods to effectively coordinate multiple resources such as source-charge-storage in frequency regulation to deal with the frequency deviation caused by random fluctuations of source-charge. In this paper, a frequency regulation algorithm for island microgrid based on the fusion of expert knowledge and deep deterministic policy gradient (DDPG) is proposed to solve the above problems. The algorithm guides the efficient interaction between the regulatory equipment and the environment through the empirical rules of expert knowledge and improve the performance of multi-resource collaborative frequency adjustment. The simulation results show that the frequency regulation strategy can make full use of multiple resources in the microgrid for frequency regulation and effectively improve the frequency regulation performance.

Key words: island microgrid, quadratic frequency modulation, expert knowledge, deep determined policy gradient

黄堃, 付明, 梁加本. 基于融合专家知识DDPG的孤岛微电网频率调节策略[J]. 中国电力, 2024, 57(2): 194-201.

Kun HUANG, Ming FU, Jiaben LIANG. Frequency Regulation Strategy of Isolated Island Microgrid Based on Fusion Expert Knowledge DDPG[J]. Electric Power, 2024, 57(2): 194-201.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202302031

https://www.electricpower.com.cn/CN/Y2024/V57/I2/194

图/表 10

参考文献 21

1	朱鹏程, 刘曌煜, 孙可, 等. 基于多分块交替方向乘子法的蜂巢状配电网分布式优化调度[J]. 中国电力, 2023, 56 (6): 90- 100.
	ZHU Pengcheng, LIU Zhaoyu, SUN Ke, et al. Optimal scheduling of honeycomb distribution network based on BADMM[J]. Electric Power, 2023, 56 (6): 90- 100.
2	涂春鸣, 杨义, 兰征, 等. 含多虚拟同步发电机的微电网二次调频策略[J]. 电工技术学报, 2018, 33 (10): 2186- 2195.
	TU Chunming, YANG Yi, LAN Zheng, et al. Secondary frequency regulation strategy in microgrid based on VSG[J]. Transactions of China Electrotechnical Society, 2018, 33 (10): 2186- 2195.
3	ZHANG T J, YUE D, YU L, et al. Joint energy and workload scheduling for fog-assisted multimicrogrid systems: a deep reinforcement learning approach[J]. IEEE Systems Journal, 2023, 17 (1): 164- 175. DOI
4	李忠文, 吴龙, 程志平, 等. 参与微电网频率调节的光伏发电系统模糊自适应功率控制[J]. 电机与控制学报, 2023, 27 (9): 126- 138.
	LI Zhongwen, WU Long, CHENG Zhiping, et al. Fuzzy adaptive power control for photovoltaic power generation participating in frequency regulation of microgrid[J]. Electric Machines and Control, 2023, 27 (9): 126- 138.
5	胡俊葵, 张益轩, 李迁, 等. 光伏发电主动参与电网频率调节控制策略研究[J]. 自动化应用, 2023, 64 (3): 53- 55. DOI
	HU Junkui, ZHANG Yixuan, LI Qian, et al. Research on control strategy of photovoltaic power generation actively participating in frequency regulation of power grid[J]. Automation Application, 2023, 64 (3): 53- 55. DOI
6	张靖, 张志文, 胡斯佳, 等. 独立微电网风储协同调频的功率柔性分配策略[J]. 电工技术学报, 2022, 37 (15): 3767- 3780.
	ZHANG Jing, ZHANG Zhiwen, HU Sijia, et al. A flexible power distribution strategy with wind turbine generator and energy storage for frequency regulation in isolated microgrid[J]. Transactions of China Electrotechnical Society, 2022, 37 (15): 3767- 3780.
7	刘起兴, 和识之, 卢伟辉, 等. 电池储能辅助二次调频的模型预测控制方法[J]. 电测与仪表, 2020, 57 (23): 119- 125.
	LIU Qixing, HE Shizhi, LU Weihui, et al. Model predictive control method for battery energy storage assisting secondary frequency regulation[J]. Electrical Measurement & Instrumentation, 2020, 57 (23): 119- 125.
8	刘进辉, 郑岩, 郭翠双. 基于负荷综合需求响应的多能互补微电网频率控制策略分析[J]. 电子技术, 2023, 52 (10): 14- 15.
	LIU Jinhui, ZHENG Yan, GUO Cuishuang. Analysis of frequency control strategy for multi energy complementary microgrid based on load comprehensive demand response[J]. Electronic Technology, 2023, 52 (10): 14- 15.
9	胡苏南, 施永, 王新颖. 基于数据驱动的孤岛微网自适应调频策略[J]. 电源学报, 2020, 18 (6): 5- 11.
	HU Sunan, SHI Yong, WANG Xinying. Adaptive frequency regulation strategy for islanded microgrid based on data driven method[J]. Journal of Power Supply, 2020, 18 (6): 5- 11.
10	王岳, 杨国华, 庄家懿, 等. 基于一致性算法的微电网无差调频控制策略[J]. 中国电力, 2020, 53 (10): 187- 191.
	WANG Yue, YANG Guohua, ZHUANG Jiayi, et al. Zero-error frequency regulation control method for microgrids based on consensus algorithm[J]. Electric Power, 2020, 53 (10): 187- 191.
11	KANDASAMY J, RAMACHANDRAN R, VEERASAMY V, et al. Distributed leader-follower based adaptive consensus control for networked microgrids[J]. Applied Energy, 2024, 353, 122083. DOI
12	ZHANG J, PAN L, HAN Q L, et al. Deep learning based attack detection for cyber-physical system cybersecurity: a survey[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9 (3): 377- 391. DOI
13	沈珺, 柳伟, 李虎成, 等. 基于强化学习的多微电网分布式二次优化控制[J]. 电力系统自动化, 2020, 44 (5): 198- 206. DOI
	SHEN Jun, LIU Wei, LI Hucheng, et al. Reinforcement learning based distributed secondary optimal control for multiple microgrids[J]. Automation of Electric Power Systems, 2020, 44 (5): 198- 206. DOI
14	HE X T, GE S Y, LIU H, et al. Frequency regulation of multi-microgrid with shared energy storage based on deep reinforcement learning[J]. Electric Power Systems Research, 2023, 214, 108962. DOI
15	张华强, 牟晨东, 赵玫, 等. 基于强化学习的多光储虚拟同步机频率协调控制策略[J]. 电气传动, 2021, 51 (19): 36- 42.
	ZHANG Huaqiang, MU Chendong, ZHAO Mei, et al. Frequency coordination control strategy of multiple photovoltaic-battery virtual synchronous generators based on reinforcement learning[J]. Electric Drive, 2021, 51 (19): 36- 42.
16	YAN R D, WANG Y, XU Y, et al. A multiagent quantum deep reinforcement learning method for distributed frequency control of islanded microgrids[J]. IEEE Transactions on Control of Network Systems, 2022, 9 (4): 1622- 1632. DOI
17	郭小龙, 张江飞, 亢朋朋, 等. 含基于PI控制受端二次调频的特高压直流虚拟同步控制策略[J]. 中国电力, 2022, 55 (11): 66- 72.
	GUO Xiaolong, ZHANG Jiangfei, KANG Pengpeng, et al. Virtual synchronization control strategy for UHVDC with secondary frequency modulation based on PI control[J]. Electric Power, 2022, 55 (11): 66- 72.
18	李清. 基于分散强化学习算法的虚拟发电部落AGC功率动态分配[D]. 广州: 华南理工大学, 2017.
	LI Qing. Distributed reinforcement learning algorithm for dynamic generation dispatch of automatic generation control based on virtual generation tribe[D]. Guangzhou: South China University of Technology, 2017.
19	张茂林, 肖辉, 曾林俊, 等. 计及储能损耗和功率预测误差的风光储微电网功率平抑策略[J]. 电器与能效管理技术, 2022, (5): 47- 54.
	ZHANG Maolin, XIAO Hui, ZENG Linjun, et al. Power smoothing strategy for wind PV energy storage microgrid considering energy storage losses and power prediction errors[J]. Electrical & Energy Management Technology, 2022, (5): 47- 54.
20	黄帅博, 陈蓓, 高降宇. 基于马尔可夫决策过程的电动汽车充电站能量管理策略[J]. 电力自动化设备, 2022, 42 (10): 92- 99.
	HUANG Shuaibo, CHEN Bei, GAO Jiangyu. Energy management strategy of electric vehicle charging station based on Markov decision process[J]. Electric Power Automation Equipment, 2022, 42 (10): 92- 99.
21	XIA Y, XU Y, WANG Y, et al. A safe policy learning-based method for decentralized and economic frequency control in isolated networked-microgrid systems[J]. IEEE Transactions on Sustainable Energy, 2022, 13 (4): 1982- 1993. DOI

参数		数值
汽轮机发电时间常数 H_r		0.4
系统惯性时间常数 M		10
阻尼系数 D		2
光伏发电时间常数 H_L		0.45
蓄电池充放电频率响应系数 H_SOC		0.4
变频空调增益 I_air		1
变频空调频率响应系数 H_air		15
电动汽车充放电频率响应系数 H_EV		11
汽轮机单位调节成本系数 ρ_G		0.45
储能折损成本系数 C₁		0.24
电动汽车调节成本系数 ρ_EV		0.45
学习率 l		0.0003
折扣因子 γ		0.95
目标动作噪声方差		0.01
经验池大小		240000

参数		数值
汽轮机发电时间常数 H_r		0.4
系统惯性时间常数 M		10
阻尼系数 D		2
光伏发电时间常数 H_L		0.45
蓄电池充放电频率响应系数 H_SOC		0.4
变频空调增益 I_air		1
变频空调频率响应系数 H_air		15
电动汽车充放电频率响应系数 H_EV		11
汽轮机单位调节成本系数 ρ_G		0.45
储能折损成本系数 C₁		0.24
电动汽车调节成本系数 ρ_EV		0.45
学习率 l		0.0003
折扣因子 γ		0.95
目标动作噪声方差		0.01
经验池大小		240000

调频性能目标权重系数φ₃	调频成本目标权重系数φ₁	调频时间目标权重系数φ₂	频率偏差$ \Delta f $
1000	$ \left(\mathrm{0,1}\right) $	$ \left(\mathrm{0,1}\right) $	$ \left(-\mathrm{0.2,0.2}\right) $
1000	$ \left(\mathrm{1,10}\right) $	$ \left(\mathrm{1,10}\right) $	$ \left(-\mathrm{0.5,0.5}\right) $
1000	$ \left(\mathrm{10,100}\right) $	$ \left(\mathrm{10,100}\right) $	$ \left(-\mathrm{0.8,0.8}\right) $

调频性能目标权重系数φ₃	调频成本目标权重系数φ₁	调频时间目标权重系数φ₂	频率偏差$ \Delta f $
1000	$ \left(\mathrm{0,1}\right) $	$ \left(\mathrm{0,1}\right) $	$ \left(-\mathrm{0.2,0.2}\right) $
1000	$ \left(\mathrm{1,10}\right) $	$ \left(\mathrm{1,10}\right) $	$ \left(-\mathrm{0.5,0.5}\right) $
1000	$ \left(\mathrm{10,100}\right) $	$ \left(\mathrm{10,100}\right) $	$ \left(-\mathrm{0.8,0.8}\right) $

[1]	孙佳航, 王小华, 黄景光, 曹豪, 梅诺男, 李浙栋. 基于MPC-VSG的孤岛微电网频率和电压动态稳定控制策略[J]. 中国电力, 2023, 56(6): 51-60,81.
[2]	张淑兴, 马驰, 杨志学, 王尧, 吴昊, 任洲洋. 基于深度确定性策略梯度算法的风光储系统联合调度策略[J]. 中国电力, 2023, 56(2): 68-76.
[3]	林日晖, 陈友立. 基于深度Q学习的含用户侧储能微电网频率-电压数字化智能控制策略[J]. 中国电力, 2022, 55(12): 43-50.
[4]	吴强, 韩俊, 金颋, 蔡超, 孟诗语, 殷俊平. 冷热电联供型孤岛微电网环保经济调度模型[J]. 中国电力, 2021, 54(12): 137-142.
[5]	凌婵惠, 郑常宝, 胡存刚, 芮涛. 考虑储能充放电效益的孤岛微电网经济优化模型[J]. 中国电力, 2019, 52(6): 111-120.
[6]	王志杰, 彭梁, 朱晓星, 陈厚涛. 基于专家知识库的双进双出制粉系统自启停技术[J]. 中国电力, 2018, 51(1): 133-138.