Frequency Regulation Strategy of Isolated Island Microgrid Based on Fusion Expert Knowledge DDPG

doi:10.11930/j.issn.1004-9649.202302031

Abstract

Abstract:

With the wide access of intermittent renewable energy such as wind and PV to island microgrid, it is difficult for traditional control methods to effectively coordinate multiple resources such as source-charge-storage in frequency regulation to deal with the frequency deviation caused by random fluctuations of source-charge. In this paper, a frequency regulation algorithm for island microgrid based on the fusion of expert knowledge and deep deterministic policy gradient (DDPG) is proposed to solve the above problems. The algorithm guides the efficient interaction between the regulatory equipment and the environment through the empirical rules of expert knowledge and improve the performance of multi-resource collaborative frequency adjustment. The simulation results show that the frequency regulation strategy can make full use of multiple resources in the microgrid for frequency regulation and effectively improve the frequency regulation performance.

Key words: island microgrid, quadratic frequency modulation, expert knowledge, deep determined policy gradient

Kun HUANG, Ming FU, Jiaben LIANG. Frequency Regulation Strategy of Isolated Island Microgrid Based on Fusion Expert Knowledge DDPG[J]. Electric Power, 2024, 57(2): 194-201.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.electricpower.com.cn/EN/10.11930/j.issn.1004-9649.202302031

https://www.electricpower.com.cn/EN/Y2024/V57/I2/194

Figures/Tables 10

References 21

1	朱鹏程, 刘曌煜, 孙可, 等. 基于多分块交替方向乘子法的蜂巢状配电网分布式优化调度[J]. 中国电力, 2023, 56 (6): 90- 100.
	ZHU Pengcheng, LIU Zhaoyu, SUN Ke, et al. Optimal scheduling of honeycomb distribution network based on BADMM[J]. Electric Power, 2023, 56 (6): 90- 100.
2	涂春鸣, 杨义, 兰征, 等. 含多虚拟同步发电机的微电网二次调频策略[J]. 电工技术学报, 2018, 33 (10): 2186- 2195.
	TU Chunming, YANG Yi, LAN Zheng, et al. Secondary frequency regulation strategy in microgrid based on VSG[J]. Transactions of China Electrotechnical Society, 2018, 33 (10): 2186- 2195.
3	ZHANG T J, YUE D, YU L, et al. Joint energy and workload scheduling for fog-assisted multimicrogrid systems: a deep reinforcement learning approach[J]. IEEE Systems Journal, 2023, 17 (1): 164- 175. DOI
4	李忠文, 吴龙, 程志平, 等. 参与微电网频率调节的光伏发电系统模糊自适应功率控制[J]. 电机与控制学报, 2023, 27 (9): 126- 138.
	LI Zhongwen, WU Long, CHENG Zhiping, et al. Fuzzy adaptive power control for photovoltaic power generation participating in frequency regulation of microgrid[J]. Electric Machines and Control, 2023, 27 (9): 126- 138.
5	胡俊葵, 张益轩, 李迁, 等. 光伏发电主动参与电网频率调节控制策略研究[J]. 自动化应用, 2023, 64 (3): 53- 55. DOI
	HU Junkui, ZHANG Yixuan, LI Qian, et al. Research on control strategy of photovoltaic power generation actively participating in frequency regulation of power grid[J]. Automation Application, 2023, 64 (3): 53- 55. DOI
6	张靖, 张志文, 胡斯佳, 等. 独立微电网风储协同调频的功率柔性分配策略[J]. 电工技术学报, 2022, 37 (15): 3767- 3780.
	ZHANG Jing, ZHANG Zhiwen, HU Sijia, et al. A flexible power distribution strategy with wind turbine generator and energy storage for frequency regulation in isolated microgrid[J]. Transactions of China Electrotechnical Society, 2022, 37 (15): 3767- 3780.
7	刘起兴, 和识之, 卢伟辉, 等. 电池储能辅助二次调频的模型预测控制方法[J]. 电测与仪表, 2020, 57 (23): 119- 125.
	LIU Qixing, HE Shizhi, LU Weihui, et al. Model predictive control method for battery energy storage assisting secondary frequency regulation[J]. Electrical Measurement & Instrumentation, 2020, 57 (23): 119- 125.
8	刘进辉, 郑岩, 郭翠双. 基于负荷综合需求响应的多能互补微电网频率控制策略分析[J]. 电子技术, 2023, 52 (10): 14- 15.
	LIU Jinhui, ZHENG Yan, GUO Cuishuang. Analysis of frequency control strategy for multi energy complementary microgrid based on load comprehensive demand response[J]. Electronic Technology, 2023, 52 (10): 14- 15.
9	胡苏南, 施永, 王新颖. 基于数据驱动的孤岛微网自适应调频策略[J]. 电源学报, 2020, 18 (6): 5- 11.
	HU Sunan, SHI Yong, WANG Xinying. Adaptive frequency regulation strategy for islanded microgrid based on data driven method[J]. Journal of Power Supply, 2020, 18 (6): 5- 11.
10	王岳, 杨国华, 庄家懿, 等. 基于一致性算法的微电网无差调频控制策略[J]. 中国电力, 2020, 53 (10): 187- 191.
	WANG Yue, YANG Guohua, ZHUANG Jiayi, et al. Zero-error frequency regulation control method for microgrids based on consensus algorithm[J]. Electric Power, 2020, 53 (10): 187- 191.
11	KANDASAMY J, RAMACHANDRAN R, VEERASAMY V, et al. Distributed leader-follower based adaptive consensus control for networked microgrids[J]. Applied Energy, 2024, 353, 122083. DOI
12	ZHANG J, PAN L, HAN Q L, et al. Deep learning based attack detection for cyber-physical system cybersecurity: a survey[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9 (3): 377- 391. DOI
13	沈珺, 柳伟, 李虎成, 等. 基于强化学习的多微电网分布式二次优化控制[J]. 电力系统自动化, 2020, 44 (5): 198- 206. DOI
	SHEN Jun, LIU Wei, LI Hucheng, et al. Reinforcement learning based distributed secondary optimal control for multiple microgrids[J]. Automation of Electric Power Systems, 2020, 44 (5): 198- 206. DOI
14	HE X T, GE S Y, LIU H, et al. Frequency regulation of multi-microgrid with shared energy storage based on deep reinforcement learning[J]. Electric Power Systems Research, 2023, 214, 108962. DOI
15	张华强, 牟晨东, 赵玫, 等. 基于强化学习的多光储虚拟同步机频率协调控制策略[J]. 电气传动, 2021, 51 (19): 36- 42.
	ZHANG Huaqiang, MU Chendong, ZHAO Mei, et al. Frequency coordination control strategy of multiple photovoltaic-battery virtual synchronous generators based on reinforcement learning[J]. Electric Drive, 2021, 51 (19): 36- 42.
16	YAN R D, WANG Y, XU Y, et al. A multiagent quantum deep reinforcement learning method for distributed frequency control of islanded microgrids[J]. IEEE Transactions on Control of Network Systems, 2022, 9 (4): 1622- 1632. DOI
17	郭小龙, 张江飞, 亢朋朋, 等. 含基于PI控制受端二次调频的特高压直流虚拟同步控制策略[J]. 中国电力, 2022, 55 (11): 66- 72.
	GUO Xiaolong, ZHANG Jiangfei, KANG Pengpeng, et al. Virtual synchronization control strategy for UHVDC with secondary frequency modulation based on PI control[J]. Electric Power, 2022, 55 (11): 66- 72.
18	李清. 基于分散强化学习算法的虚拟发电部落AGC功率动态分配[D]. 广州: 华南理工大学, 2017.
	LI Qing. Distributed reinforcement learning algorithm for dynamic generation dispatch of automatic generation control based on virtual generation tribe[D]. Guangzhou: South China University of Technology, 2017.
19	张茂林, 肖辉, 曾林俊, 等. 计及储能损耗和功率预测误差的风光储微电网功率平抑策略[J]. 电器与能效管理技术, 2022, (5): 47- 54.
	ZHANG Maolin, XIAO Hui, ZENG Linjun, et al. Power smoothing strategy for wind PV energy storage microgrid considering energy storage losses and power prediction errors[J]. Electrical & Energy Management Technology, 2022, (5): 47- 54.
20	黄帅博, 陈蓓, 高降宇. 基于马尔可夫决策过程的电动汽车充电站能量管理策略[J]. 电力自动化设备, 2022, 42 (10): 92- 99.
	HUANG Shuaibo, CHEN Bei, GAO Jiangyu. Energy management strategy of electric vehicle charging station based on Markov decision process[J]. Electric Power Automation Equipment, 2022, 42 (10): 92- 99.
21	XIA Y, XU Y, WANG Y, et al. A safe policy learning-based method for decentralized and economic frequency control in isolated networked-microgrid systems[J]. IEEE Transactions on Sustainable Energy, 2022, 13 (4): 1982- 1993. DOI

参数		数值
汽轮机发电时间常数 H_r		0.4
系统惯性时间常数 M		10
阻尼系数 D		2
光伏发电时间常数 H_L		0.45
蓄电池充放电频率响应系数 H_SOC		0.4
变频空调增益 I_air		1
变频空调频率响应系数 H_air		15
电动汽车充放电频率响应系数 H_EV		11
汽轮机单位调节成本系数 ρ_G		0.45
储能折损成本系数 C₁		0.24
电动汽车调节成本系数 ρ_EV		0.45
学习率 l		0.0003
折扣因子 γ		0.95
目标动作噪声方差		0.01
经验池大小		240000

参数		数值
汽轮机发电时间常数 H_r		0.4
系统惯性时间常数 M		10
阻尼系数 D		2
光伏发电时间常数 H_L		0.45
蓄电池充放电频率响应系数 H_SOC		0.4
变频空调增益 I_air		1
变频空调频率响应系数 H_air		15
电动汽车充放电频率响应系数 H_EV		11
汽轮机单位调节成本系数 ρ_G		0.45
储能折损成本系数 C₁		0.24
电动汽车调节成本系数 ρ_EV		0.45
学习率 l		0.0003
折扣因子 γ		0.95
目标动作噪声方差		0.01
经验池大小		240000

调频性能目标权重系数φ₃	调频成本目标权重系数φ₁	调频时间目标权重系数φ₂	频率偏差$ \Delta f $
1000	$ \left(\mathrm{0,1}\right) $	$ \left(\mathrm{0,1}\right) $	$ \left(-\mathrm{0.2,0.2}\right) $
1000	$ \left(\mathrm{1,10}\right) $	$ \left(\mathrm{1,10}\right) $	$ \left(-\mathrm{0.5,0.5}\right) $
1000	$ \left(\mathrm{10,100}\right) $	$ \left(\mathrm{10,100}\right) $	$ \left(-\mathrm{0.8,0.8}\right) $

调频性能目标权重系数φ₃	调频成本目标权重系数φ₁	调频时间目标权重系数φ₂	频率偏差$ \Delta f $
1000	$ \left(\mathrm{0,1}\right) $	$ \left(\mathrm{0,1}\right) $	$ \left(-\mathrm{0.2,0.2}\right) $
1000	$ \left(\mathrm{1,10}\right) $	$ \left(\mathrm{1,10}\right) $	$ \left(-\mathrm{0.5,0.5}\right) $
1000	$ \left(\mathrm{10,100}\right) $	$ \left(\mathrm{10,100}\right) $	$ \left(-\mathrm{0.8,0.8}\right) $

[1]	Xinghua HUANG, Han WU, Shichuan CHEN, Yuanliang FAN, Gonglin ZHANG. An Optimization Method for Energy Storage Configuration of Isolated Island Microgrid Considering New Energy Output [J]. Electric Power, 2024, 57(12): 132-138.
[2]	LIN Rihui, CHEN Youli. Frequency-Voltage Digital Intelligent Control Strategy of Microgrid with User-Side Energy Storage Based on Deep Q-Learning [J]. Electric Power, 2022, 55(12): 43-50.
[3]	LING Chanhui, ZHENG Changbao, HU Cungang, RUI Tao. Real-Time Economic Optimization Method for Microgrid Considering Energy Storage Charge and Discharge Benefits [J]. Electric Power, 2019, 52(6): 111-120.
[4]	WANG Zhijie, PENG Liang, ZHU Xiaoxing, CHEN Houtao. The Automatic Startup and Shutdown Technology for Double Inlet and Outlet Coal-pulverizing System Based on Expert Knowledge Base [J]. Electric Power, 2018, 51(1): 133-138.