Energy storage capacity configuration method based on deep reinforcement learning and multi-objective particle swarm optimization

MOU Xuepeng; CAI Quan; ZHOU Yiran; LI Qingsheng; ZHANG Yu

doi:10.19799/j.cnki.2095-4239.2025.0915

您当前的位置：

首页 >

文章列表页 >

Energy storage capacity configuration method based on deep reinforcement learning and multi-objective particle swarm optimization

Energy Storage System and Engineering | 更新时间：2026-05-28

- Energy storage capacity configuration method based on deep reinforcement learning and multi-objective particle swarm optimization
- Energy Storage Science and Technology Vol. 15, Issue 2, Pages: 583-590(2026)
- 作者机构：
  
  1.贵州电网公司电网规划研究中心，贵州贵阳 550002
  2.贵州省新型电力系统运行控制全省重点实验室，贵州贵阳 550000
  3.贵州电网公司电网规划研究中心人才工作站，贵州贵阳 550002
- 作者简介：
- 基金信息：
- DOI：10.19799/j.cnki.2095-4239.2025.0915
  CLC： TM 715;TM 912
- Received：14 October 2025，
  
  Revised：2025-11-27，
  
  Published：28 February 2026
- 稿件说明：
移动端阅览
牟雪鹏, 蔡权, 周依然, 等. 基于深度强化学习和多目标粒子群优化的储能容量配置方法[J]. 储能科学与技术, 2026, 15(2): 583-590.

MOU Xuepeng, CAI Quan, ZHOU Yiran, et al. Energy storage capacity configuration method based on deep reinforcement learning and multi-objective particle swarm optimization[J]. Energy Storage Science and Technology, 2026, 15(2): 583-590.
牟雪鹏, 蔡权, 周依然, 等. 基于深度强化学习和多目标粒子群优化的储能容量配置方法[J]. 储能科学与技术, 2026, 15(2): 583-590. DOI： 10.19799/j.cnki.2095-4239.2025.0915.

MOU Xuepeng, CAI Quan, ZHOU Yiran, et al. Energy storage capacity configuration method based on deep reinforcement learning and multi-objective particle swarm optimization[J]. Energy Storage Science and Technology, 2026, 15(2): 583-590. DOI： 10.19799/j.cnki.2095-4239.2025.0915.

摘要

随着新能源大规模并网，其出力的间歇性与随机性给电力系统稳定运行带来挑战，储能合理容量配置对提升系统可靠性、促进新能源消纳至关重要。本工作提出基于深度强化学习(DRL)与多目标粒子群优化(MOPSO)的储能容量配置方法：首先分析电力系统运行特性与储能接入需求，构建含风光、负荷的系统模型，明确配置约束与多目标优化目标；其次利用DRL学习复杂环境下储能运行策略，通过DRL智能体与电网环境交互输出充放电时序控制建议，将该建议作为MOPSO的粒子初始化边界与搜索方向引导，结合MOPSO对储能容量组合的全局搜索能力，实现多目标寻优过程中经济性、可靠性与新能源消纳目标的冲突平衡。以实际电网为例的案例分析表明，该方法相较于传统方法，能在满足系统约束的情况下，优化储能配置方案，提升经济性、可靠性与新能源消纳能力，为大规模新能源接入下的储能规划提供可行的技术手段，助力电力系统低碳高效转型。

Abstract

With the increasing penetration of renewable energy sources into power grids

the inherent intermittency and randomness of power generation pose substantial operational challenges to maintaining the stability of power systems. The optimal capacity configuration of energy storage systems plays a critical role in enhancing system reliability and facilitating the efficient integration and utilization of renewable energy. This paper develops an energy storage capacity configuration framework based on the combined application of deep reinforcement learning (DRL) and multiobjective particle swarm optimization (MOPSO). First

the proposed framework systematically analyzes the operational characteristics of power systems and the grid-integration requirements of energy storage

and subsequently constructs a comprehensive system model incorporating wind power

solar power

and time-varying load demand

while explicitly defining configuration constraints and multiobjective optimization objectives. Second

DRL is employed to learn adaptive energy storage operational strategies under complex and uncertain environments; through continuous interaction between the DRL agent and the power grid environment

the agent derives optimal charging and discharging schedules. These learned operational strategies are then used to define particle initialization bounds and guide the search direction of the MOPSO algorithm. By leveraging MOPSO's global search capability over energy storage capacity combinations

the proposed framework effectively coordinates and balances inherently conflicting objectives

including economic performance

system reliability

and renewable energy utilization

within a unified multiobjective optimization process. Case studies based on an actual power grid demonstrate that

compared with traditional methods

the proposed approach achieves superior energy storage configuration solutions while satisfying system constraints

resulting in marked improvements in economic efficiency

system reliability

and renewable energy utilization capacity. Overall

the proposed framework provides a practical and scalable technical solution for energy storage planning under large-scale renewable energy integration and supports the low-carbon

reliable

and efficient transition of modern power systems.

关键词

Keywords

references

陈昀, 刘继雷, 刘蓓, 等. 基于不确定性天气数据的光伏发电系统电量分时段预测[J]. 中国测试, 2023, 49(S2): 76-83.

CHEN Yun, LIU Jilei, LIU Bei, et al. Timephased prediction of photovoltaic power generation system based on uncertain weather data[J]. China Measurement & Test, 2023, 49(S2): 76-83.

周校聿, 刘娆, 鲍福增, 等. 百兆瓦级储能参与电网双重辅助服务调度的联合优化模型[J]. 电力系统自动化, 2021, 45 (19): 60-69.

ZHOU Xiaoyu, LIU Rao, BAO Fuzeng, et al. Joint optimization model for hundred-megawatt-level energy storage participating in dual ancillary services dispatch of power grid[J]. Automation of Electric Power Systems, 2021, 45(19): 60-69.

闫震, 刘强, 李会彬, 等. 基于混合储能荷电状态的光伏微网功率优化管理方法[J]. 储能科学与技术, 2025, 14(5): 2067-2077.

YAN Z, LIU Q, LI H B, et al. Power optimization management method for photovoltaic microgrids based on the state of charge of hybrid energy storage systems[J]. Energy Storage Science and Technology, 2025, 14(5): 2067-2077.

朱兰, 董凯旋, 唐陇军, 等. 计及同步机惯性与储能虚拟惯性价值的电能、惯性及一次调频联合优化出清模型 [J]. 中国电机工程学报, 2024, 44 (19): 7543-7554. DOI:10.13334/j.0258-8013.pcsee.230306.

ZHU Lan, DONG Kaixuan, TANG Longjun, et al. Joint optimal clearing model for electric energy,inertia and primary frequency response considering synchronous inertia and energy storage virtual inertia values[J]. Proceedings of the CSEE, 2024, 44(19): 7543-7554. DOI:10.13334/j.0258-8013.pcsee.230306.

柳长发, 付立衡, 张增丽, 等. 高比例光伏接入的分布式储能容量自适应协调控制方法[J]. 储能科学与技术, 2024, 13(8): 2696-2703.

LIU C F, FU L H, ZHANG Z L, et al. Adaptive coordinated control method for distributed energy storage capacity with high proportion of photovoltaic access[J]. Energy Storage Science and Technology, 2024, 13(8): 2696-2703.

商立群, 闵鹏波, 张建涛. 基于改进PSO算法的光伏阵列MPPT研究[J]. 传感器与微系统, 2024, 43(8): 35-39.

SHANG Liqun, MIN Pengbo, ZHANG Jiantao. Research on MPPT of photovoltaic array based on improved PSO algorithm[J]. Transducer and Microsystem Technologies, 2024, 43(8): 35-39.

刘凡, 李凤婷, 张高航, 等. 计及循环寿命和运营策略的风电汇集区域储能电站优化配置[J]. 电力系统保护与控制, 2023, 51(8): 127-139. DOI: 10.19783/j.cnki.pspc.221188.

LIU F, LI F T, ZHANG G H, et al. Optimal configuration of storage power stations in a wind power gathering area considering cycle life and operation strategy[J]. Power System Protection and Control, 2023, 51(8): 127-139. DOI: 10.19783/j.cnki.pspc.221188.

阙凌燕, 蒋正威, 杨力强, 等.基于改进遗传算法的源网荷储协同控制方法[J]. 沈阳工业大学学报, 2023, 45(6): 612-618.

QUE Lingyan, JIANG Zhengwei, YANG Liqiang,et al. Cooperative control method based on improved genetic algorithm for source network load storage[J].Journal of Shenyang University of Technology, 2023,45(6): 612-618.

马骞, 肖亮, 程冰, 等. 基于强化学习-模型预测控制(RL-MPC)的分布式储能协同一次调频控制方法[J].储能科学与技术, 2025, 14(08): 3138-3148.

MA Qian, XIAO Liang, CHENG Bing, et al. Cooperative primary frequency modulation control method for distributed energy storage based on reinforcement learning model predictive control[J]. Energy Storage Science and Technology, 2025, 14(8): 3138-3148.

周楠, 樊玮, 刘念, 等. 基于需求响应的光伏微网储能系统多目标容量优化配置[J]. 电网技术, 2016, 40(6): 1709-1716. DOI: 10.13335/j.1000-3673.pst.2016.06.015.

ZHOU N, FAN W, LIU N, et al. Battery storage multi-objective optimization for capacity configuration of PV-based microgrid considering demand response[J]. Power System Technology, 2016, 40(6): 1709-1716. DOI: 10.13335/j.1000-3673.pst. 2016. 06.015.

卢秀丽, 安娟华, 冀松, 等. 基于粒子群优化的WSN多目标3D无人机运动规划[J]. 传感器与微系统, 2025, 44(11): 153-158.

LU X L, AN J H, JI S, et al. Multi-objective 3D UAV motion planning in WSN based on article swarm optimization[J]. Transducer and Microsystem Technologies, 2025, 44(11): 153-158.

苏明健, 肖宝弟, 岳丽丽. 基于改进PSO-SA算法的城轨列车ATO节能优化研究[J]. 传感器与微系统, 2023, 42(10): 64-67, 76. DOI: 10.13873/J.1000-9787(2023)10-0064-04.

SU M J, XIAO B D, YUE L L. Research on ATO energy saving optimization of urban rail train based on improved PSO-SA algorithm[J]. Transducer and Microsystem Technologies, 2023, 42(10): 64-67, 76. DOI: 10.13873/J.1000-9787(2023)10-0064-04.

叶林, 王凯丰, 赖业宁, 等. 低惯量下电力系统频率特性分析及电池储能调频控制策略综述[J]. 电网技术, 2023, 47(2): 446-462. DOI: 10.13335/j.1000-3673.pst.2022.1269.

YE L, WANG K F, LAI Y N, et al. Review of frequency characteristics analysis and battery energy storage frequency regulation control strategies in power system under low inertia level[J]. Power System Technology, 2023, 47(2): 446-462. DOI: 10.13335/j.1000-3673.pst.2022.1269.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A grid-side energy storage system optimization method based on improved twin deep deterministic policy gradient and adaptive distributed model predictive control

Energy management and optimal scheduling strategies for energy storage systems based on deep reinforcement learning

Research on capacity configuration and energy optimization of energy storage systems in rail transit

Coordinative optimal dispatch of multi-park integrated energy system considering complementary cooling， heating and power and energy storage systems

Design of a battery cooling system enhanced by synergistic combination of TPMS and phase change materials

Related Author

CAI Quan

ZHOU Yiran

LI Qingsheng

TAN Jinlong

CHEN Jun

ZHAO Qi

CUI Dalin

LIU Yongqiang

Related Institution

State Key Laboratory of Power Transmission and Distribution Equipment and System Safety and New Technology, Chongqing University

Electric Power Research Institute, State Grid Xinjiang Electric Power Co., Ltd.

State Grid Xinjiang Electric Power Co.,Ltd.

Ganzhou Teachers College

Guangzhou Railway Polytechnic, Transportation and Logistics Industry

⁰