Cloud-Computing Based Power Big Data Analysis Technology and Its Application

doi:10.11930.2015.2.111

Abstract

Abstract: In order to deal with the bottlenecks of the performance and scalability of power data analysis in present big data era and meet the requirements of power production and marking systems, the advantages of cloud computing technology is firstly analyzed, then the architecture and key techniques of Hadoop-based power big data analysis system are proposed. The architecture combines the characteristics of power big data and the emerging cloud computing technology, and includes three key technologies, i.e., the distributed gridfile based multi-dimensional index, the automatic SQL to HiveQL translation tool and the hybrid data storage model, which can support data update operation. Therefore, the traditional power data analysis system can be improved and updated. The application of the system in the power information collection system in Zhejiang indicates that the performance of the system is improved by 5 times with only one-eighth of the hardware configuration as compared with the traditional system. This proves that the cloud computing technology can significantly improve the performance and reduce the cost of big data acquisition and analysis.

Key words: electric power, big data, cloud computing, electricity information, data acquisition, data analysis

CLC Number:

TM769

WU Kaifeng, LIU Wantao, LI Yanhu, SU Yipeng, XIAO Zheng, PEI Xubin, HU Songlin. Cloud-Computing Based Power Big Data Analysis Technology and Its Application[J]. Electric Power, 2015, 48(2): 111-116.

Add to citation manager EndNote|Ris|BibTeX

URL: https://www.electricpower.com.cn/EN/10.11930.2015.2.111

https://www.electricpower.com.cn/EN/Y2015/V48/I2/111

References

[1] 陈树勇，宋书芳，李兰欣，等. 智能电网技术综述[J]. 电网技术，2009，33（8）：1－7．
CHEN Shuyong, SONG Shufang, LI Lanxin, et al. Survey on smart grid technology [J]. Power System Technology, 2009, 33(8): 1-7.
[2] 杨方，王文迪，葛旭波，等. 我国智能电网发展格局及综合评价[J]. 中国电力，2012，45（12）：81－85.
YANG Fang, WANG Wendi, GE Xubo, et al. Comprehensive assessment on development patterns of smart grid in China [J]. Electric Power, 2012, 45(12): 81-85.
[3] 郑雅楠，单葆国，顾宇桂，等. 中长期电量灰色预测中数据预处理方法研究[J]. 中国电力，2013，46（10）：111－114．
ZHENG Yanan, SHAN Baoguo, GU Yugui, et al. Data preprocessing for grey model of medium-Long term load forecasting [J]. Electric Power, 2013, 46(10): 111-114.
[4] 中国电机工程学会信息化专委会. 中国电力大数据发展白皮书[R]. 2013.
[5] 罗军舟，金嘉晖，宋爱波，等. 云计算-体系架构与关键技术[J].通信学报，2011，32（7）：3－21．
LUO Junzhou, JIN Jiahui, SONG Aibo, et al. Cloud computing: architecture and key technologies [J]. Journal on Communications,2011, 32(7): 3-21.
[6] 曹阳，高志远，杨胜春，等. 云计算模式在电力调度系统中的应用[J]. 中国电力，2012，45（6）：14－17．
CAO Yang, GAO Zhiyuan, YANG Shengchun, et al. Application of cloud computing in power dispatching systems[J]. Electric Power, 2012,45 (6): 14-17.
[7] BORTHAKUR D. HDFS architecture guide[EB/OL]. (2013-04-04)[2014-09-04]. http://hadoop.apache.org/docs/r1.2.1/hdfs_ design.html.
[8] WHITE T，曾大聘，周傲英. Hadoop权威指南[M]. 北京：清华大学出版社，2010.
[9] LEVERENZ L. Apache Hive[EB/OL]. (2013-12-18) [2014-09-12]. https://cwiki.apache.org/confluence/display/Hive/Home.
[10] NIEVERGELT J, HINTERBERGER H, SEVCIK K C. The grid file: An adaptable, symmetric multikey file structure[J]. ACM Transactions on Database Systems, 1984, 9(1): 38-71.
[11] HADDAD J. The Apache Cassandra Project[EB/OL]. (2009-09-13) [2014-08-27]. http://cassandra.apache.org/.
[12] XU Yingzhong, HU Songlin. QMapper: a tool for SQL optimization on hive using query rewriting [C]//Proceedings of the 22nd International Conference on World Wide Web Companion. Rio de Janeiro, 2013: 211-212.

[1]	Mingrun TANG, Ruoyang LI, Muran LIU, Xiaoyu CHENG, Yao LIU, Shuxia YANG. Prediction of Large-Scale Renewable Energy Access under Steady State of Electric Power System [J]. Electric Power, 2025, 58(2): 126-132.
[2]	Xudong LI, Qingbo TAN, Haochen ZHAO, Ning QIAO, Liwei LIU, Caixia TAN, Zhongfu TAN. Carbon Emission Factors and Decoupling Effects of China's Power Industry under the Background of Carbon Peak [J]. Electric Power, 2024, 57(5): 88-98.
[3]	MA Chao. Review and Prospect of Power Standard Digitization for Machine Readable Standards [J]. Electric Power, 2023, 56(8): 216-229.
[4]	LI Da, GUO Qinglei, WANG Dong, WANG Weixian, ZHOU Dongxu. Research on Electric Power Data Verification and Traceability Method Based on Blockchain Technology [J]. Electric Power, 2023, 56(5): 203-208.
[5]	Zhilin LIU, Chuanlong XU, Haifeng ZHENG, Jiangtao LI, Li YAO, Qing LIU, Yi DU. Empirical Analysis of Ensuring Electricity Supply and Promoting Renewable Power Consumption Before and After the Shutdown of Nuclear Power Plants: the Germany Case [J]. Electric Power, 2023, 56(10): 145-152.
[6]	YU Jilai, GUO Yufeng, SHENG Yuhe, ZHANG Wei, WANG Songyan. Coupling Analysis of Energy Flow and Economic Flow in Power Grid [J]. Electric Power, 2022, 55(6): 42-52.
[7]	ZHOU Decai, CHEN Jinjin, LIU Bo. Mixed-frequency Asymmetric Measure of China's Electric Power Industry Prosperity Index [J]. Electric Power, 2022, 55(6): 172-185.
[8]	GUAN Li, ZHOU Lei, LIU Hanghang, ZHOU Xinsheng, TAO Wei, ZHAO Zizhou. Analysis of Independent Energy Storage Power Station Participating in Power Spot Market Mechanism and Trial Operation [J]. Electric Power, 2022, 55(10): 185-190.
[9]	ZHAO Bingzhen, LI Da, WANG Dong, GUO Qinglei, YU Guo. Research on Standard System of Energy and Electric Power Blockchain Technology [J]. Electric Power, 2022, 55(10): 191-200.
[10]	LIU Wenjun, DONG Ming, XU Yuanfu, HAN Qiang, WANG Xin, XU Lei, DU Ming. Structure and Key Technologies of Big Data Labeling System for Power Equipment Operation Status [J]. Electric Power, 2022, 55(1): 126-132.
[11]	BAI Wanrong, ZHANG Xun, ZHU Xiaoqin, LIU Jixiang, CHENG Qiyu, ZHAO Yan, SHAO Jie. E-FCNN Based Electric Power Inspection Image Enhancement [J]. Electric Power, 2021, 54(5): 179-185.
[12]	ZHAO Pu, ZHOU Man, GAO Jianyu, PAN Lezhen, LIU Zifa, XIE Haokai. Evaluation Method for Park-Level Integrated Energy System Based on Electric Power Substitution [J]. Electric Power, 2021, 54(4): 130-140.
[13]	XIAO Lei, LI Bozhong, ZHANG Suxiang, LAI Junsen, LIU Lu, TIAN Zhaoyu. Application and Evaluation of Quantum Secure Communication in Electric Power Communication Networks [J]. Electric Power, 2021, 54(1): 175-181,202.
[14]	ZHANG Qiaoyu, ZHANG Lan, XIAO Yunpeng, DAI Xiaojuan, LAI Xiaowen. Design of South China (Starting from Guangdong Province) Power Reserve Market [J]. Electric Power, 2020, 53(9): 28-37.
[15]	LI Bo, FANG Tong. Application and Prospect of Beidou Navigation Satellite System (BDS) in Smart Grid [J]. Electric Power, 2020, 53(8): 107-116.