基于多智能体深度确定策略梯度算法的火力发电商竞价策略

doi:10.11930/j.issn.1004-9649.202309119

中国电力 ›› 2024, Vol. 57 ›› Issue (11): 161-172.DOI: 10.11930/j.issn.1004-9649.202309119

基于多智能体深度确定策略梯度算法的火力发电商竞价策略

张兴平¹(), 王腾¹(), 张馨月¹(), 张浩楠²()

1. 华北电力大学经济与管理学院，北京　102206
2. 华北电力大学（保定）经济与管理系，河北保定　071003

收稿日期:2023-09-25 接受日期:2024-02-20 出版日期:2024-11-28 发布日期:2024-11-27
作者简介:张兴平（1972—），男，通信作者，博士，教授，从事电力市场研究，E-mail：zxp@ncepu.edu.cn
王腾（1998—），男，硕士研究生，从事电力市场、智能报价研究，E-mail：471692144@qq.com
张馨月（1996—），女，博士研究生，从事碳市场、绿证市场研究，E-mail：15895965970@163.com
张浩楠（1992—），男，博士，讲师，从事电力转型与电力市场研究，E-mail： zhanghn2022@ncepu.edu.cn
基金资助:
国家自然科学基金资助项目（促进我国发电侧高效清洁发展的多市场耦合优化与机制研究，72074075）；国家社会科学基金重大项目（“双碳”目标下能源结构转型路径与协同机制研究，22ZD104）。

Bidding Strategy for Thermal Power Generation Companies Based on Multi-agent Deep Deterministic Policy Gradient Algorithm

Xingping ZHANG¹(), Teng WANG¹(), Xinyue ZHANG¹(), Haonan ZHANG²()

1. School of Economics and Management, North China Electric Power University, Beijing 102206, China
2. Department of Economic Management, North China Electric Power University, Baoding 071003, China

Received:2023-09-25 Accepted:2024-02-20 Online:2024-11-28 Published:2024-11-27
Supported by:
This work is supported by the National Natural Science Foundation of China (Research on Multi-market Coupling Optimization and Mechanism for High-efficiency and Clean Development of Power Generation in China, No.72074075) and Major Program of the National Social Science Foundation of China ("Double Carbon" Target and Pathways for Energy Structure Transformation and Synergistic Mechanisms, No.22ZD104).

摘要/Abstract

摘要：

火电是新型电力系统的重要支撑，研究火力发电商的竞价策略以及不同出清机制的影响，对保障其低碳高效运营具有重要意义。构建基于多智能体深度确定策略梯度算法的竞价策略模型，分析不同火力发电商组合的竞价差异化策略，优化多主体报价报量策略，探究边际统一出清、按报价支付出清和随机匹配出清3种典型出清机制的市场影响。结果表明，该策略模型可引导火力发电商采取合理的竞价方式以提高市场效率；在新能源渗透率较低时，不同出清机制对各类型机组的影响有所不同；随着新能源渗透率的提高，采用按报价支付出清机制可以兼顾经济和环境效益；当新能源渗透率达到较高水平时，采用随机匹配出清机制可有效应对市场波动风险。

关键词: 火力发电商, 多智能体, 出清机制, 竞价策略, 新能源渗透率

Abstract:

Thermal power is an important support for the new power system. It is of great significance to study the bidding strategy for thermal power generation companies and the influence of different clearing mechanisms to ensure their low-carbon and efficient operation. A bidding strategy model is constructed based on the multi-agent deep deterministic policy gradient algorithm to analyze the differential bidding strategies for different combinations of thermal power generation companies. The multi-agent price and quantity bidding strategy is optimized, and the market impact of different market clearing mechanisms is explored. The simulation results indicate that the proposed bidding strategy model can guide the thermal power generation companies to optimize their bidding methods and improve the market efficiency. When the penetration rate of new energy is low, the applicability of different clearing mechanisms varies for various types of units; with the increase of the penetration rate of new energy, the pay as bid mechanism can be used to enhance the economic and environmental efficiency of the electricity market; when the penetration rate of new energy reaches a high level, the random matching clearing mechanism can effectively address market volatility risks.

Key words: thermal power generation companies, multi-agent, clearing mechanism, bidding strategy, new energy penetration

张兴平, 王腾, 张馨月, 张浩楠. 基于多智能体深度确定策略梯度算法的火力发电商竞价策略[J]. 中国电力, 2024, 57(11): 161-172.

Xingping ZHANG, Teng WANG, Xinyue ZHANG, Haonan ZHANG. Bidding Strategy for Thermal Power Generation Companies Based on Multi-agent Deep Deterministic Policy Gradient Algorithm[J]. Electric Power, 2024, 57(11): 161-172.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202309119

https://www.electricpower.com.cn/CN/Y2024/V57/I11/161

图/表 13

图 1 月度集中市场撮合竞价过程

Fig.1 Matching bidding process in the monthly centralized market

图 2 基于多智能体深度确定性策略梯度算法下的发电商竞价模型框架

Fig.2 Bidding model framework for power generation companies based on MADDPG

表 1 发电商技术参数

Table 1 Technical parameter of power generators

编号	机组容量/MW	a	b	c	可申报电量/(MW·h)	边际成本/ (元·(kW·h)^–1）
G1	330	0.00140	–0.8391	484.2	46012.9	0.3877
G2	330	0.00140	–0.8439	495.6	46012.9	0.3960
G3	330	0.00130	–0.8700	477.8	48801.6	0.3465
G4	330	0.00080	–0.5428	416.6	47407.3	0.3119
G5	330	0.00090	–0.5910	417.0	92025.8	0.3210
G6	330	0.00090	–0.6375	422.3	92025.8	0.2956
G7	660	0.00030	–0.3747	413.8	92025.8	0.3112
G8	600	0.00030	–0.3532	437.3	83659.9	0.3375
G9	660	0.00020	–0.3241	430.8	92025.8	0.2643
G10	600	0.00030	–0.3384	439.3	83659.9	0.3572
G11	1000	0.00005	–0.1208	356.1	139433.1	0.2645
G12	1000	0.00010	–0.2442	420.5	147799.1	0.2320

表 2 不同智能体组合方案

Table 2 Different agent combination schemes

机组容量/ MW	设置发电机组为智能体
机组容量/ MW	S1	S2	S3	S4	S5	S6	S7	S8
300	√	×	×	√	√	×	√	×
600	×	√	×	√	×	√	√	×
1000	×	×	√	×	√	√	√	×

表 3 排放收益率

Table 3 Emission return rate of different clearing mechanisms

出清机制	排放收益率/(元·t^–1)
出清机制	S1	S2	S3	S4	S5	S6	S7	S8
1	0.154	0.146	0.168	0.183	0.187	0.151	0.228	0.137
2	0.141	0.154	0.094	0.219	0.186	0.125	0.227	0.092
3	0.128	0.140	0.146	0.181	0.163	0.17	0.213	0.117

图 3 不同智能体组合下市场出清结果对比

Fig.3 Comparison of market clearing results under different agent combinations

图 4 报价支付出清机制下机组报价及报量策略系数

Fig.4 Strategic coefficient of prices and quantities for units under pay as bid mechanism

图 5 机组出清结果对比

Fig.5 Comparison of unit clearing results

图 6 部分机组供电煤耗率曲线

Fig.6 Coal consumption rate curve for power supply of some units

图 7 按报价支付出清机制下各类型机组的收益变化

Fig.7 Changes in revenue for various types of units under pay as bid mechanism

图 8 不同出清机制下市场效率对比

Fig.8 Comparison of market efficiency under different clearing mechanisms

图 9 不同新能源渗透率下出清结果变化

Fig.9 Changes in clearing results under different new energy penetration rates

表 4 不同新能源渗透率下排放收益率

Table 4 Unit emission return rate under different new energy penetration rates

出清机制	不同新能源渗透率下排放收益
出清机制	10%	20%	30%	40%	50%	60%
按报价支付出清	0.215	0.206	0.203	0.195	0.165	0.162
边际统一出清	0.214	0.201	0.172	0.126	0.107	0.107
随机匹配出清	0.208	0.204	0.191	0.182	0.177	0.171

参考文献 31

1	王林, 李晨, 刘嘉佳, 等. 基于复式竞价撮合的电力市场交易模式设计与实践[J]. 电力系统自动化, 2018, 42 (24): 188- 195.
	WANG Lin, LI Chen, LIU Jiajia, et al. Design and practice of electricity market trading mode based on compound bidding matchmaking[J]. Automation of Electric Power Systems, 2018, 42 (24): 188- 195.
2	黄远明, 卢恩, 赖晓文, 等. 广东月度集中竞争交易机制设计与实践[J]. 南方电网技术, 2018, 12 (8): 51- 58.
	HUANG Yuanming, LU En, LAI Xiaowen, et al. Design and practice of monthly centralized trading mechanism in Guangdong Province[J]. Southern Power System Technology, 2018, 12 (8): 51- 58.
3	程乐峰, 余涛. 发电市场长期竞价均衡自发形成过程中的一般多策略演化博弈决策行为研究[J]. 中国电机工程学报, 2020, 40 (21): 6936- 6955.
	CHENG Lefeng, YU Tao. Decision-making behavior investigation for general multi-strategy evolutionary games in the spontaneous formation of long-term bidding equilibria of A power generation market[J]. Proceedings of the CSEE, 2020, 40 (21): 6936- 6955.
4	何洋, 黄龙, 陈皓勇, 等. 基于协同进化算法的集中竞价市场模拟分析[J]. 浙江电力, 2019, 38 (7): 7- 13.
	HE Yang, HUANG Long, CHEN Haoyong, et al. Simulation analysis of centralized bidding market based on co-evolutionary algorithm[J]. Zhejiang Electric Power, 2019, 38 (7): 7- 13.
5	齐世雄, 王秀丽, 张炜, 等. 基于博弈论和改进PageRank的电力市场出清方式评价方法[J]. 电力建设, 2019, 40 (5): 107- 117. DOI
	QI Shixiong, WANG Xiuli, ZHANG Wei, et al. Evaluation method for electricity market clearing model based on game theory and improved PageRank[J]. Electric Power Construction, 2019, 40 (5): 107- 117. DOI
6	石可, 陈皓勇, 李鹏, 等. 基于协同进化的两种电力市场出清机制分析[J]. 电力系统自动化, 2019, 43 (9): 68- 74.
	SHI Ke, CHEN Haoyong, LI Peng, et al. Analysis on two kinds of electricity market clearance mechanism based on co-evolution[J]. Automation of Electric Power Systems, 2019, 43 (9): 68- 74.
7	陈皓勇, 付超. 统一价格和PAB竞价的实验分析[J]. 电力系统自动化, 2007, 31 (4): 12- 17.
	CHEN Haoyong, FU Chao. Experimental analysis of uniform price and PAB auctions in electricity markets[J]. Automation of Electric Power Systems, 2007, 31 (4): 12- 17.
8	HAN D, SUN M. The design of a Probability Bidding Mechanism in electricity auctions by considering trading constraints[J]. Simulation, 2015, 91 (10): 916- 924. DOI
9	LIU Z, ZHANG X L, LIEU J. Design of the incentive mechanism in electricity auction market based on the signaling game theory[J]. Energy, 2010, 35 (4): 1813- 1819. DOI
10	薛贵元, 吴垠, 诸晓骏, 等. 计及用能权配额约束的发电商竞价策略双层优化方法[J]. 中国电力, 2023, 56 (5): 51- 61.
	XUE Guiyuan, WU Yin, ZHU Xiaojun, et al. Bi-level optimization method for bidding strategy of power suppliers considering energy-consuming right[J]. Electric Power, 2023, 56 (5): 51- 61.
11	田福银, 马骏, 王灿, 等. 基于双层主从博弈的综合能源系统多主体低碳经济运行策略[J]. 中国电力, 2022, 55 (11): 184- 193.
	TIAN Fuyin, MA Jun, WANG Can, et al. Multi-agent low-carbon and economy operation strategy of integrated energy system based on Bi-level master-slave game[J]. Electric Power, 2022, 55 (11): 184- 193.
12	杨朋朋, 王蓓蓓, 胥鹏, 等. 不完全信息下基于深度双Q网络的发电商三段式竞价策略[J]. 中国电力, 2021, 54 (11): 47- 58.
	YANG Pengpeng, WANG Beibei, XU Peng, et al. Three-stage bidding strategy of generation company based on double deep Q-network under incomplete information condition[J]. Electric Power, 2021, 54 (11): 47- 58.
13	NAMALOMBA E, HU F H, SHI H J. Agent based simulation of centralized electricity transaction market using bi-level and Q-learning algorithm approach[J]. International Journal of Electrical Power & Energy Systems, 2022, 134, 107415.
14	高宇, 李昀, 曹蓉蓉, 等. 基于多代理Double DQN算法模拟发电侧竞价行为[J]. 电网技术, 2020, 44 (11): 4175- 4182.
	GAO Yu, LI Yun, CAO Rongrong, et al. Simulation of generators' bidding behavior based on multi-agent double DQN[J]. Power System Technology, 2020, 44 (11): 4175- 4182.
15	周翔, 王继业, 陈盛, 等. 基于深度强化学习的微网优化运行综述[J]. 全球能源互联网, 2023, 6 (3): 240- 257.
	ZHOU Xiang, WANG Jiye, CHEN Sheng, et al. Review of microgrid optimization operation based on deep reinforcement learning[J]. Journal of Global Energy Interconnection, 2023, 6 (3): 240- 257.
16	徐尔丰. 基于A3C强化学习的电力市场发电商报价策略研究[D]. 北京: 华北电力大学(北京), 2019.
	XU Erfeng. Research on bidding strategy of generators in electricity market based on asynchronous advantage actor-critic reinforcement learning[J]. Beijing: North China Electric Power University(Beijing), 2019.
17	马丽莹, 魏云冰. 基于DDPG算法的发电企业报价策略研究[J]. 电气工程学报, 2023, 18 (2): 192- 200.
	MA Liying, WEI Yunbing. Research on bidding strategy of power generation enterprise based on DDPG algorithm[J]. Journal of Electrical Engineering, 2023, 18 (2): 192- 200.
18	LIANG Y C, GUO C L, DING Z H, et al. Agent-based modeling in electricity market using deep deterministic policy gradient algorithm[J]. IEEE Transactions on Power Systems, 2020, 35 (6): 4180- 4192. DOI
19	YE Y J, QIU D W, SUN M Y, et al. Deep reinforcement learning for strategic bidding in electricity markets[J]. IEEE Transactions on Smart Grid, 2020, 11 (2): 1343- 1355. DOI
20	YE Y J, QIU D W, LI J, et al. Multi-period and multi-spatial equilibrium analysis in imperfect electricity markets: a novel multi-agent deep reinforcement learning approach[J]. IEEE Access, 2019, 7, 130515- 130529. DOI
21	DU Y, LI F X, ZANDI H L, et al. Approximating Nash equilibrium in day-ahead electricity market bidding with multi-agent deep reinforcement learning[J]. Journal of Modern Power Systems and Clean Energy, 2021, 9 (3): 534- 544. DOI
22	LOWE R, WU Y, TAMAR A, et al. Multi-agent actor-critic for mixed cooperative-competitive environments[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA. ACM, 2017: 6382–6393.
23	于申, 申建建, 程春田, 等. 耦合梯级水电调蓄价值的月度集中撮合交易出清方法[J]. 中国电机工程学报, 2022, 42 (16): 5858- 5868.
	YU Shen, SHEN Jianjian, CHENG Chuntian, et al. Centralized matchmaking transaction clearing method embedded the regulation value of cascade hydropower[J]. Proceedings of the CSEE, 2022, 42 (16): 5858- 5868.
24	张兴平, 何澍, 王泽嘉, 等. 不同新能源渗透率下燃煤机组行为策略分析[J]. 电力建设, 2022, 43 (5): 9- 17.
	ZHANG Xingping, HE Shu, WANG Zejia, et al. Behavior strategy of coal-fired units under different new energy penetration rate[J]. Electric Power Construction, 2022, 43 (5): 9- 17.
25	ZHANG X Y, GUO X P, ZHANG X P. Collaborative strategy within China's emission trading scheme: evidence from a tripartite evolutionary game model[J]. Journal of Cleaner Production, 2023, 382, 135255. DOI
26	李姚旺, 刘昱良, 杨晓斌, 等. 计及电量交易信息的用电碳计量方法[J]. 中国电机工程学报, 2024, 44 (2): 439- 451.
	LI Yaowang, LIU Yuliang, YANG Xiaobin, et al. Electricity carbon metering method considering electricity transaction information[J]. Proceedings of the CSEE, 2024, 44 (2): 439- 451.
27	万里鹏, 兰旭光, 张翰博, 等. 深度强化学习理论及其应用综述[J]. 模式识别与人工智能, 2019, 32 (1): 67- 81.
	WAN Lipeng, LAN Xuguang, ZHANG Hanbo, et al. A review of deep reinforcement learning theory and application[J]. Pattern Recognition and Artificial Intelligence, 2019, 32 (1): 67- 81.
28	赵立阳, 常天庆, 褚凯轩, 等. 完全合作类多智能体深度强化学习综述[J]. 计算机工程与应用, 2023, 59 (12): 14- 27. DOI
	ZHAO Liyang, CHANG Tianqing, CHU Kaixuan, et al. Survey of fully cooperative multi-agent deep reinforcement learning[J]. Computer Engineering and Applications, 2023, 59 (12): 14- 27. DOI
29	王军, 曹雷, 陈希亮, 等. 多智能体博弈强化学习研究综述[J]. 计算机工程与应用, 2021, 57 (21): 1- 13. DOI
	WANG Jun, CAO Lei, CHEN Xiliang, et al. Overview on reinforcement learning of multi-agent game[J]. Computer Engineering and Applications, 2021, 57 (21): 1- 13. DOI
30	李鹏, 黄文琦, 王鑫, 等. 数据与知识联合驱动的人工智能方法在电力调度中的应用综述[J]. 电力系统自动化, 2024, 48 (1): 160- 175. DOI
	LI Peng, HUANG Wenqi, WANG Xin, et al. Review on application of combined data-knowledge-driven artificial intelligence methods in power dispatching[J]. Automation of Electric Power Systems, 2024, 48 (1): 160- 175. DOI
31	刘福国, 蒋学霞, 李志. 燃煤发电机组负荷率影响供电煤耗的研究[J]. 电站系统工程, 2008, 24 (4): 47- 49.
	LIU Fuguo, JIANG Xuexia, LI Zhi. Investigation on affects of generator load on coal consumption rate in fossil power plant[J]. Power System Engineering, 2008, 24 (4): 47- 49.

[1]	曾仪, 周毅, 陆继翔, 周良才, 唐宁恺, 李红. 基于多智能体安全深度强化学习的电压控制[J]. 中国电力, 2025, 58(2): 111-117.
[2]	李超英, 檀勤良. 基于智能体建模的新型电力系统下火电企业市场交易策略[J]. 中国电力, 2024, 57(2): 212-225.
[3]	薛贵元, 吴垠, 诸晓骏, 谈健, 明昊. 计及用能权配额约束的发电商竞价策略双层优化方法[J]. 中国电力, 2023, 56(5): 51-61.
[4]	余运俊, 杨升宝, 闵卫东, 余光华, 韦伯鲁. 双链存储的微电网分级电能交易策略[J]. 中国电力, 2022, 55(4): 156-165.
[5]	丁雨, 于艾清, 高纯. 基于改进一致性算法的独立光储直流微电网电压稳定能量协调策略[J]. 中国电力, 2022, 55(3): 74-79.
[6]	李咸善, 陈奥博, 程杉, 陈敏睿. 基于生态博弈的含云储能微电网多智能体协调优化调度[J]. 中国电力, 2021, 54(7): 166-177.
[7]	杨朋朋, 王蓓蓓, 胥鹏, 王高琴, 郑亚先. 不完全信息下基于深度双Q网络的发电商三段式竞价策略[J]. 中国电力, 2021, 54(11): 47-58.
[8]	赵凤贤，吴静，孙丽颖. 基于MAPSO优化的智能配电网大面积断电供电恢复[J]. 中国电力, 2016, 49(1): 85-90.

基于多智能体深度确定策略梯度算法的火力发电商竞价策略

Bidding Strategy for Thermal Power Generation Companies Based on Multi-agent Deep Deterministic Policy Gradient Algorithm

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 31

相关文章 8

编辑推荐

Metrics

模态框（Modal）标题

基于多智能体深度确定策略梯度算法的火力发电商竞价策略

Bidding Strategy for Thermal Power Generation Companies Based on Multi-agent Deep Deterministic Policy Gradient Algorithm

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 31

相关文章 8

编辑推荐

Metrics