Self-evolving Power Smooth Control Method for Offshore Wind Power Cluster Based On Deep Reinforcement Learning

doi:10.11930/j.issn.1004-9649.202206099

Abstract

Abstract: The offshore wind speed has a high spatial-temporal correlation, which aggravates the power fluctuation of the whole wind farm and poses significant challenges to the power system, especially when large-scale offshore wind power is integrated. Smoothing control of large-scale offshore wind power clusters is a key solution to mitigating the above problems. However, most existing methods are inefficient and difficult to support higher frequency control and are susceptible to wind power forecast errors and the deviation of actual action from the optimal control command. Therefore, this paper proposes a new control framework for “offline-training, online-optimization and self-evolution”, and establishes a deep-reinforcement-learning-based model for the smoothing control of the active power of offshore wind power clusters. Firstly, a short-term revenue function for cluster power smoothing control is proposed to solve the optimal command based on the Markov decision process model. Secondly, a long-term revenue policy function for power policy calibration is proposed to effectively correct the control deviation according to the historical feedback data. Finally, a deep neural network model is established for the mapping between the state of the agent, the control benefit and the control decision to realize the training and solution of the agent based on the deep deterministic policy gradient algorithm. The results show that the proposed method can reduce the power fluctuation by 20% and control the power loss within 5% under the given wind condition of 7.5 m/s average wind speed.

Key words: wind power cluster, active power control, deep reinforcement learning, fluctuation smoothing, control deviation correction

SONG Weiye, LIU Lingyue, YAN Jie, WANG Hangyu, HE Shukai, HAN Shuang, WANG Minghui, LIU Yongqian. Self-evolving Power Smooth Control Method for Offshore Wind Power Cluster Based On Deep Reinforcement Learning[J]. Electric Power, 2023, 56(3): 36-46.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.electricpower.com.cn/EN/10.11930/j.issn.1004-9649.202206099

https://www.electricpower.com.cn/EN/Y2023/V56/I3/36

References

[1] 时智勇, 王彩霞, 李琼慧. “十四五”中国海上风电发展关键问题[J]. 中国电力, 2020, 53(7): 8–17
SHI Zhiyong, WANG Caixia, LI Qionghui. Key issues of China's offshore wind power development in the “14 th five-year plan”[J]. Electric Power, 2020, 53(7): 8–17
[2] 房方, 梁栋炀, 刘亚娟, 等. 海上风电智能控制与运维关键技术[J]. 发电技术, 2022, 43(2): 175–185
FANG Fang, LIANG Dongyang, LIU Yajuan, et al. Key technologies for intelligent control and operation and maintenance of offshore wind power[J]. Power Generation Technology, 2022, 43(2): 175–185
[3] 林俐, 朱晨宸, 郑太一, 等. 风电集群有功功率控制及其策略[J]. 电力系统自动化, 2014, 38(14): 9–16
LIN Li, ZHU Chenchen, ZHENG Taiyi, et al. Active power control of wind farm cluster and its strategy[J]. Automation of Electric Power Systems, 2014, 38(14): 9–16
[4] 鲁宗相, 叶一达, 郭莉, 等. 电力电子化电力系统的调频挑战与多层级协调控制框架[J]. 中国电力, 2019, 52(4): 8–17,110
LU Zongxiang, YE Yida, GUO Li, et al. Frequency regulation challenge of power electronics dominated power systems and its new multi-level coordinated control framework[J]. Electric Power, 2019, 52(4): 8–17,110
[5] 樊新东, 杨秀媛, 金鑫城, 等. 风电场有功功率控制综述[J]. 发电技术, 2018, 39(3): 268–276
FAN Xindong, YANG Xiuyuan, JIN Xincheng. An overview of active power control in wind farms[J]. Power Generation Technology, 2018, 39(3): 268–276
[6] 吴涛, 刘立红, 王岱岚, 等. 风电场智能化远程集控系统设计[J]. 中国电力, 2018, 51(4): 161–167
WU Tao, LIU Lihong, WANG Dailan. Design of remote centralized intelligent monitoring system for wind farms[J]. Electric Power, 2018, 51(4): 161–167
[7] YE L, ZHANG C H, TANG Y, et al. Hierarchical model predictive control strategy based on dynamic active power dispatch for wind power cluster integration[J]. IEEE Transactions on Power Systems, 2019, 34(6): 4617–4629.
[8] 施贵荣, 孙荣富, 徐海翔, 等. 大规模集群可再生能源有功分层协调控制策略[J]. 电网技术, 2018, 42(7): 2160–2167
SHI Guirong, SUN Rongfu, XU Haixiang, et al. Active power stratification coordination control strategy for large-scale cluster of renewable energy[J]. Power System Technology, 2018, 42(7): 2160–2167
[9] MA S K, GENG H, YANG G, et al. Clustering-based coordinated control of large-scale wind farm for power system frequency support[J]. IEEE Transactions on Sustainable Energy, 2018, 9(4): 1555–1564.
[10] KHALID M, SAVKIN A V. A model predictive control approach to the problem of wind power smoothing with controlled battery storage[J]. Renewable Energy, 2009, 35(7): 1520–1526.
[11] 鹿婷, 贾继超, 彭晓涛. 一种考虑经济调度的风电场储能控制策略[J]. 分布式能源, 2019, 4(3): 40–49
LU Ting, JIA Jichao, PENG Xiaotao. An energy storage control strategy for wind farm considering economic dispatching[J]. Distributed Energy, 2019, 4(3): 40–49
[12] 席裕庚, 李德伟, 林姝. 模型预测控制-现状与挑战[J]. 自动化学报, 2013, 39(3): 222–236
XI Yugeng, LI Dewei, LIN Shu. Model predictive control-status and challenges[J]. Acta Automatica Sinica, 2013, 39(3): 222–236
[13] GAO X D, MENG K, DONG Z Y, et al. Cooperation-driven distributed control scheme for large-scale wind farm active power regulation[J]. IEEE Transactions on Energy Conversion, 2017, 32(3): 1240–1250.
[14] 武晗, 鲁宗相, 白恺, 等. 风光储联合电站有功控制的响应迟滞“拖尾现象”分析与改进[J]. 电网技术, 2017, 41(4): 1068–1076
WU Han, LU Zongxiang, BAI Kai, et al. Analysis and optimization of ‘tailing phenomenon’ in active power control of hybrid wind-PV-ES power system[J]. Power System Technology, 2017, 41(4): 1068–1076
[15] 黄鹏翔, 周云海, 徐飞, 等. 基于灵活性裕度的含风电电力系统源荷储协调滚动调度[J]. 中国电力, 2020, 53(11): 78–88
HUANG Pengxiang, ZHOU Yunhai, XU Fei, et al. Source-load-storage coordinated rolling dispatch for wind power integrated power system based on flexibility margin[J]. Electric Power, 2020, 53(11): 78–88
[16] 仲悟之, 李梓锋, 肖洋, 等. 高渗透联网风电集群有功分层递阶控制策略[J]. 电网技术, 2018, 42(6): 1868–1875
ZHONG Wuzhi, LI Zifeng, XIAO Yang, et al. Active hierarchical progressive control strategy of highly penetrated networked wind power cluster[J]. Power System Technology, 2018, 42(6): 1868–1875
[17] 张斌, 左剑, 周年光, 等. 考虑变桨机组动态响应的风电场闭环有功控制方法[J]. 电力系统及其自动化学报, 2017, 29(7): 136–143
ZHANG Bin, ZUO Jian, ZHOU Nianguang, et al. Closed-loop active power control method for wind farm considering the dynamic response of variable-pitch wind turbine[J]. Proceedings of the CSU-EPSA, 2017, 29(7): 136–143
[18] 张晋华, 朱悦榕, 李旭强, 等. 考虑风电不确定性的含海上风电场电力系统优化调度策略研究[J]. 分布式能源, 2021, 6(5): 33–43
ZHANG Jinhua, ZHU Yuerong, LI Xuqiang, et al. Optimization scheduling strategy of offshore wind farm power system considering wind power uncertainty[J]. Distributed Energy, 2021, 6(5): 33–43
[19] 葛晓琳, 郝广东, 夏澍, 等. 高比例风电系统的优化调度方法[J]. 电网技术, 2019, 43(2): 390–400
GE Xiaolin, HAO Guangdong, XIA Shu, et al. An optimal system scheduling method with high proportion of wind power[J]. Power System Technology, 2019, 43(2): 390–400
[20] 朱瑛, 高云波, 臧海祥, 等. 风电机组输出功率平滑技术综述[J]. 电力系统自动化, 2018, 42(18): 182–191
ZHU Ying, GAO Yunbo, ZANG Haixiang, et al. Review of output power smoothing technologies for wind turbine[J]. Automation of Electric Power Systems, 2018, 42(18): 182–191
[21] 刘军, 张彬彬, 赵晨聪. 基于数据驱动的风电场有功功率分配算法[J]. 电力系统自动化, 2019, 43(17): 125–131
LIU Jun, ZHANG Binbin, ZHAO Chencong. Data-driven based active power distribution algorithm in wind farm[J]. Automation of Electric Power Systems, 2019, 43(17): 125–131
[22] 路朋, 叶林, 裴铭, 等. 风电集群有功功率模型预测协调控制策略[J]. 中国电机工程学报, 2021, 41(17): 5887–5900
LU Peng, YE Lin, PEI Ming, et al. Coordinated control strategy for active power of wind power cluster based on model predictive control[J]. Proceedings of the CSEE, 2021, 41(17): 5887–5900
[23] 陈晓光, 杨秀媛, 王镇林, 等. 考虑多目标优化模型的风电场储能容量配置方案[J]. 发电技术, 2022, 43(5): 718–730
CHEN Xiaoguang, YANG Xiuyuan, WANG Zhenlin, et al. Energy storage capacity allocation scheme of wind farm considering multi-objective optimization model[J]. Power System Technology:, 2022, 43(5): 718–730
[24] SULTANA W R, SAHOO S K, SUKCHAI S, et al. A review on state of art development of model predictive control for renewable energy applications[J]. Renewable & sustainable energy reviews, 2017, 76: 391–406.
[25] LIAO S Y, XU J, SUN Y Z, et al. Control of energy-intensive load for power smoothing in wind power plants[J]. IEEE Transactions on Power Systems, 2018, 33(6): 6142–6154.
[26] 张中泉, 刘晓光, 钟天宇. 风电机组有功功率集散优化控制系统研究[J]. 发电技术, 2018, 39(6): 574–579
ZHANG Zhongquan, LIU Xiaoguang, ZHONG Tianyu. Research on optimal distributed control system of active power of wind turbine generators[J]. Power Generation Technology, 2018, 39(6): 574–579
[27] YE Y J, QIU D W, SUN M Y, et al. Deep reinforcement learning for strategic bidding in electricity markets[J]. IEEE Transactions on Smart Grid, 2020, 11(2): 1343–1355.
[28] 张淑兴, 马驰, 杨志学, 等. 基于深度确定性策略梯度算法的风光储系统联合调度策略[J/OL]. 中国电力: 1–9[2022-06-08].
ZHANG Shuxing, MA Chi, YANG Zhixue, et al. Joint dispatch of wind-photovoltaic-storage hybrid system based on deep deterministic policy gradient algorithm [J/OL]. Electric Power: 1–9[2022-06-08].
[29] 孙宏斌, 黄天恩, 郭庆来, 等. 基于仿真大数据的电网智能型超前安全预警技术[J]. 南方电网技术, 2016, 10(3): 42–46
SUN Hongbin, HUANG Tianen, GUO Qinglai, et al. Power grid intelligent security early warning technology based on big simulation data[J]. Southern Power System Technology, 2016, 10(3): 42–46
[30] LIN Z W, CHEN Z, QU C Z, et al. A hierarchical clustering-based optimization strategy for active power dispatch of large-scale wind farm[J]. International Journal of Electrical Power & Energy Systems, 2020, 121: 106155–106163.