Optimal Management of Renewable Energy Certificates: A Reinforcement Learning Approach

Problem definition: The renewable energy certificate (REC) market plays a critical role in ensuring renewable energy compliance by facilitating the matching of REC supply and demand. In this study, we focus on the problem faced by REC aggregators who act as brokers, making REC sales decisions on behalf of individual renewable energy generators. Methodology/results: We formulate the aggregator’s REC management problem as a discrete-time Markov decision process (MDP) and derive structural properties of the optimal policy for three prevalent service contracts in the marketplace. We develop a solution framework based on a deep reinforcement learning (DRL) algorithm that integrates state-of-the-art features, including dueling double deep Q-networks (DDDQNs) and graph neural networks (GNNs). Our solution algorithm leverages the structural properties of the problem to enhance the efficiency and performance of REC management. Managerial implications: Our simulation study, based on data from two representative U.S. REC markets, highlights the role and value of REC aggregation services for small-scale generators. Finally, we provide further insights into how REC markets and services perform under various business conditions.

Funding: D. G. Choi acknowledges the support by H Energy Co. Ltd. M. K. Lim acknowledges the support by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea [Grant NRF-2022S1A5A2A01038230], as well as the Institute of Management Research, College of Business Administration, Seoul National University.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.0477.