Application-Driven Learning: A Closed-Loop Prediction and Optimization Approach Applied to Dynamic Reserves and Demand Forecasting

References

• Amazon (2024) EC2 Amazon Web Services, AWS. Accessed March 18, 2024, https://aws.amazon.com/ec2/.Google Scholar
• Babaeinejadsarookolaee S, Birchfield A, Christie RD, Coffrin C, DeMarco C, Diao R, Ferris M, et al. (2019) The power grid library for benchmarking ac optimal power flow algorithms. Preprint, submitted August 7, https://arxiv.org/abs/1908.02788.Google Scholar
• Ban GY, Gallien J, Mersereau AJ (2019) Dynamic procurement of new products with covariate information: The residual tree method. Manufacturing Service Oper. Management 21(4):798–815.Link, Google Scholar
• Bengio Y (1997) Using a financial training criterion rather than a prediction criterion. International J. Neural Systems 8(4):433–443.Crossref, Google Scholar
• Bertsimas D, Kallus N (2020) From predictive to prescriptive analytics. Management Sci. 66(3):1025–1044.Link, Google Scholar
• Billingsley P (1986) Probability and Measure, Wiley Series in Probability and Statistics (Wiley, New York).Google Scholar
• Böhm V (1975) On the continuity of the optimal policy set for linear programs. SIAM J. Appl. Math. 28(2):303–306.Crossref, Google Scholar
• Borrelli F, Bemporad A, Morari M (2003) Geometric algorithm for multiparametric linear programming. J. Optim. Theory Appl. 118(3):515–540.Crossref, Google Scholar
• Brockwell PJ, Davis RA (2009) Time Series: Theory and Methods (Springer Science & Business Media, New York).Google Scholar
• Bucksteeg M, Niesen L, Weber C (2016) Impacts of dynamic probabilistic reserve sizing techniques on reserve requirements and system costs. IEEE Trans. Sustain. Energy 7(4):1408–1420.Crossref, Google Scholar
• CAISO (2020) 2019 Annual report on market issues and performance. Department of Market Monitoring, California Independent System Operator. Accessed March 18, 2024, http://www.caiso.com/Documents/2019AnnualReportonMarketIssuesandPerformance.pdf.Google Scholar
• Chen Y, Gribik P, Gardner J (2013) Incorporating post zonal reserve deployment transmission constraints into energy and ancillary service co-optimization. IEEE Trans. Power Systems 29(2):537–549.Crossref, Google Scholar
• Conn AR, Scheinberg K, Vicente LN (2009) Introduction to Derivative-Free Optimization (SIAM, Philadelphia).Google Scholar
• De Vos K, Stevens N, Devolder O, Papavasiliou A, Hebb B, Matthys-Donnadieu J (2019) Dynamic dimensioning approach for operating reserves: Proof of concept in Belgium. Energy Policy 124:272–285.Crossref, Google Scholar
• Diao S, Sen S (2024) Distribution-free algorithms for predictive stochastic programming in the presence of streaming data. Comput. Optim. Appl. 87(2):355–395.Crossref, Google Scholar
• Dias Garcia J, Bodin G, Street A (2023) BilevelJuMP.jl: Modeling and solving bilevel optimization problems in Julia. INFORMS J. Comput. 36(2):327–335.Link, Google Scholar
• Domowitz I, El-Gamal MA (1993) A consistent test of stationary-ergodicity. Econometric Theory 9(4):589–601.Crossref, Google Scholar
• Donti P, Amos B, Kolter JZ (2017) Task-based end-to-end model learning in stochastic optimization. Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, eds. Advances in Neural Information Processing Systems, vol. 30 (Curran Associates, Inc., Red Hook, NY), 1–11.Google Scholar
• EIA (2024) U.S. Energy Information Administration, 2021. Accessed March 18, 2024, https://www.eia.gov/electricity/gridmonitor/dashboard/electric_overview/balancing_authority/PJM.Google Scholar
• Ela E, Milligan M, Kirby B (2011) Operating reserves and variable generation. Technical Report NREL/TP-5500-51978, National Renewable Energy Laboratory (NREL), Golden, CO. Accessed March 18, 2024, https://www.nrel.gov/docs/fy11osti/51978.pdf.Google Scholar
• Elmachtoub AN, Grigas P (2022) Smart “predict, then optimize”. Management Sci. 68(1):9–26.Link, Google Scholar
• Fliscounakis S, Panciatici P, Capitanescu F, Wehenkel L (2013) Contingency ranking with respect to overloads in very large power systems taking into account uncertainty, preventive, and corrective actions. IEEE Trans. Power Systems 28(4):4909–4917.Crossref, Google Scholar
• Fortuny-Amat J, McCarl B (1981) A representation and economic interpretation of a two-level programming problem. J. Oper. Res. Soc. 32(9):783–792.Crossref, Google Scholar
• Gal T (1994) Postoptimal Analyses, Parametric Programming, and Related Topics: Degeneracy, Multicriteria Decision Making, Redundancy (Walter de Gruyter, Berlin).Crossref, Google Scholar
• Grigas P, Qi M, Shen ZJM (2021) Integrated conditional estimation-optimization. Preprint, submitted October 24, https://arxiv.org/abs/2110.12351.Google Scholar
• Gurobi Optimization (2020) Gurobi optimizer reference manual. Accessed March 18, 2024, http://www.gurobi.com.Google Scholar
• Holttinen H, Milligan M, Ela E, Menemenlis N, Dobschinski J, Rawn B, Bessa RJ, Flynn D, Gomez-Lazaro E, Detlefsen NK (2012) Methodologies to determine operating reserves due to increased wind power. IEEE Trans. Sustain. Energy 3(4):713–723.Crossref, Google Scholar
• Josz C, Fliscounakis S, Maeght J, Panciatici P (2016) AC power flow data in MATPOWER and QCQP format: iTesla, RTE snapshots, and PEGASE. Preprint, submitted March 4, https://arxiv.org/abs/1603.01533.Google Scholar
• Kannan R, Bayraksan G, Luedtke J (2022) Data-driven sample average approximation with covariate information. Preprint, submitted July 27, https://arxiv.org/abs/2207.13554.Google Scholar
• Kao Yh, Roy B, Yan X (2009) Directed regression. Bengio Y, Schuurmans D, Lafferty J, Williams C, Culotta A, eds. Advances in Neural Information Processing Systems, vol. 22 (Curran Associates, Inc., Red Hook, NY), 1–9.Google Scholar
• Kirschen DS, Strbac G (2018) Fundamentals of Power System Economics (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Korf LA, Wets RJB (2001) Random LSC functions: An ergodic theorem. Math. Oper. Res. 26(2):421–445.Link, Google Scholar
• Lubin M, Dowson O, Dias Garcia J, Huchette J, Legat B, Vielma JP (2023) JuMP 1.0: Recent improvements to a modeling language for mathematical optimization. Math. Programming Comput. 15(3):581–589.Crossref, Google Scholar
• Mandi J, Bucarey V, Tchomba MMK, Guns T (2022) Decision-focused learning: Through the lens of learning to rank. Chaudhuri K, Jegelka S, Song L, Szepesvari C, Niu G, Sabato S, eds. Proc. 39th Internat. Conf. Machine Learning, vol. 162 (PMLR, New York), 14935–14947.Google Scholar
• Megiddo N, Chandrasekaran R (1989) On the ε-perturbation method for avoiding degeneracy. Oper. Res. Lett. 8(6):305–308.Crossref, Google Scholar
• Mogensen PK, Riseth AN (2018) Optim: A mathematical optimization package for Julia. J. Open Source Software 3(24):615.Crossref, Google Scholar
• Muñoz M, Pineda S, Morales J (2022) A bilevel framework for decision-making under uncertainty with contextual information. Omega 108:102575.Crossref, Google Scholar
• Nocedal J, Wright S (2006) Numerical Optimization (Springer Science & Business Media, New York).Google Scholar
• Papavasiliou A, Oren SS (2013) Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network. Oper. Res. 61(3):578–592.Link, Google Scholar
• Papavasiliou A, Oren SS, Rountree B (2014) Applying high performance computing to transmission-constrained stochastic unit commitment for renewable energy integration. IEEE Trans. Power Systems 30(3):1109–1120.Crossref, Google Scholar
• Pereira MV, Granville S, Fampa MH, Dix R, Barroso LA (2005) Strategic bidding under uncertainty: A binary expansion approach. IEEE Trans. Power Systems 20(1):180–188.Crossref, Google Scholar
• PJM (2018) PJM Manual 11: Energy and Ancillary Services Market Operations (97):200. Accessed March 18, 2024, https://pjm.com/-/media/documents/manuals/archive/m11/m11v97-energy-and-ancillary-services-market-operations-07-26-2018.ashx.Google Scholar
• Rockafellar RT, Uryasev S, Zabarankin M (2008) Risk tuning with generalized linear regression. Math. Oper. Res. 33(3):712–729.Link, Google Scholar
• Sadana U, Chenreddy A, Delage E, Forel A, Frejinger E, Vidal T (2023) A survey of contextual optimization methods for decision making under uncertainty. Preprint, submitted June 17, https://arxiv.org/abs/2306.10374.Google Scholar
• Shapiro A, Xu H (2008) Stochastic mathematical programs with equilibrium constraints, modelling and sample average approximation. Optimization 57(3):395–418.Crossref, Google Scholar
• Shapiro A, Dentcheva D, Ruszczyński A (2014) Lectures on Stochastic Programming: Modeling and Theory, 2nd ed. (SIAM, Philadelphia).Crossref, Google Scholar
• Siddiqui S, Gabriel SA (2013) An SOS1-based approach for solving MPECs with a natural gas market application. Networks Spatial Econom. 13(2):205–227.Crossref, Google Scholar
• Strbac G, Shakoor A, Black M, Pudjianto D, Bopp T (2007) Impact of wind generation on the operation and development of the UK electricity systems. Electric Power Systems Res. 77(9):1214–1227.Crossref, Google Scholar
• Sur A, Birge J (2020) Epi-convergence of sample averages of a random lower semi-continuous functional generated by a Markov chain and application to stochastic optimization. Optimization Online (April 21), https://optimization-online.org/2020/04/7762/.Google Scholar
• Tibshirani R (2011) Regression shrinkage and selection via the lasso: A retrospective. J. Royal Statist. Soc. Series B Statist. Methodol. 73(3):273–282.Crossref, Google Scholar
• Varian H (1975) A Bayesian approach to real estate assessment. Fienberg SE, Zellner A, eds. Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage (North-Holland Co., Amsterdam).Google Scholar
• Wang B, Hobbs BF (2014) A flexible ramping product: Can it help real-time dispatch markets approach the stochastic dispatch ideal? Electric Power Systems Res. 109:128–140.Crossref, Google Scholar
• Wang B, Hobbs BF (2015) Real-time markets for flexiramp: A stochastic unit commitment-based analysis. IEEE Trans. Power Systems 31(2):846–860.Crossref, Google Scholar
• White H (1984) Asymptotic Theory for Econometricians (Academic Press, Orlando, FL).Google Scholar
• Zellner A (1986a) Bayesian estimation and prediction using asymmetric loss functions. J. Amer. Statist. Assoc. 81(394):446–451.Crossref, Google Scholar
• Zellner A (1986b) Biased predictors, rationality and the evaluation of forecasts. Econom. Lett. 21(1):45–48.Crossref, Google Scholar