Revisiting Approximate Linear Programming: Constraint-Violation Learning with Applications to Inventory Control and Energy Storage

References

• Adelman D (2003) Price-directed replenishment of subsets: Methodology and its application to inventory routing. Manufacturing Service Oper. Management 5(4):348–371.Link, Google Scholar
• Adelman D (2004) A price-directed approach to stochastic inventory/routing. Oper. Res. 52(4):499–514.Link, Google Scholar
• Adelman D (2007) Dynamic bid prices in revenue management. Oper. Res. 55(4):647–661.Link, Google Scholar
• Adelman D, Barz C (2013) A unifying approximate dynamic programming model for the economic lot scheduling problem. Math. Oper. Res. 39(2):374–402.Link, Google Scholar
• Adelman D, Klabjan D (2012) Computing near-optimal policies in generalized joint replenishment. INFORMS J. Comput. 24(1):148–164.Link, Google Scholar
• Adelman D, Mersereau A (2013) Dynamic capacity allocation to customers who remember past service. Management Sci. 59(3):592–612.Link, Google Scholar
• Bach E, Shallit JO (1996) Algorithmic Number Theory: Efficient Algorithms, vol. 1 (MIT Press, Boston).Google Scholar
• Benjaafar S, ElHafsi M, Huang T (2010) Optimal control of a production-inventory system with both backorders and lost sales. Naval Res. Logist. 57(3):252–265.Crossref, Google Scholar
• Bertsekas PB (2007) Dynamic Programming and Optimal Control, 3rd ed., vol. 2 (Athena Scientific, Belmont, MA).Google Scholar
• Bhat B, Farias VF, Moallemi CC (2012) Non-parametric approximate dynamic programming via the kernel method. Working paper, Columbia University, New York.Google Scholar
• Birge JR, Louveaux F (2011) Introduction to Stochastic Programming (Springer Science & Business Media, New York).Crossref, Google Scholar
• Bishop CM (2006) Pattern Recognition and Machine Learning (Springer, New York).Google Scholar
• Brown DB, Haugh MB (2017) Information relaxation bounds for infinite horizon Markov decision processes. Oper. Res. 65(5):1355–1379.Link, Google Scholar
• Brown DB, Smith JE (2014) Information relaxations, duality, and convex stochastic dynamic programs. Oper. Res. 62(6):1394–1415.Link, Google Scholar
• Brown DB, Smith JE, Sun P (2010) Information relaxations and duality in stochastic dynamic programs. Oper. Res. 58(4):785–801.Link, Google Scholar
• Bubeck S (2015) Convex optimization: Algorithms and complexity. Foundations Trends Machine Learn. 8(3–4):231–357.Crossref, Google Scholar
• Chen X, Pang Z, Pan L (2014a) Coordinating inventory control and pricing strategies for perishable products. Oper. Res. 62(2):284–300.Link, Google Scholar
• Chen Y, Lan G, Ouyang Y (2014b) Optimal primal-dual methods for a class of saddle point problems. SIAM J. Optim. 24(4):1779–1814.Crossref, Google Scholar
• de Farias DP, Van Roy B (2003) The linear programming approach to approximate dynamic programming. Oper. Res. 51(6):850–865.Link, Google Scholar
• de Farias DP, Van Roy B (2004) On constraint sampling for the linear programming approach to approximate dynamic programming. Math. Oper. Res. 29(3):462–478.Link, Google Scholar
• Desai VV, Farias VF, Moallemi CC (2012) Pathwise optimization for optimal stopping problems. Management Sci. 58(12):2292–2308.Link, Google Scholar
• Duchi J (2016) Introductory lectures on stochastic optimization. Unpublished lecture notes, Stanford University, Stanford, CA.Google Scholar
• Erseghe T, Zanella A, Codemo CG (2014) Optimal and compact control policies for energy storage units with single and multiple batteries. IEEE Trans. Smart Grid 5(3):1308–1317.Crossref, Google Scholar
• Fan K (1953) Minimax theorems. Proc. Natl. Acad. Sci. 39(1):42–47.Crossref, Google Scholar
• Farias VF, Van Roy B (2006) Tetris: A study of randomized constraint sampling. Calafiore G, Dabbene F, eds. Probabilistic and Randomized Methods for Design Under Uncertainty (Springer-Verlag, London), 189–201.Crossref, Google Scholar
• Farias VF, Van Roy B (2007) An approximate dynamic programming approach to network revenue management. Working paper, Stanford University, Stanford, CA.Google Scholar
• Grillo S, Marinelli M, Massucco S, Silvestro F (2012) Optimal management strategy of a battery-based storage system to improve renewable energy integration in distribution networks. IEEE Trans. Smart Grid 3(2):950–958.Crossref, Google Scholar
• Hazan E, Kale S (2014) Beyond the regret minimization barrier: Optimal algorithms for stochastic strongly-convex optimization. J. Machine Learn. Res. 15(1):2489–2512.Google Scholar
• Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov Control Processes: Basic Optimality Criteria, vol. 30 (Springer, New York).Crossref, Google Scholar
• Juditsky A, Nemirovski A (2011a) First-order methods for nonsmooth convex large-scale optimization, I: General purpose methods. Sra S, Nowozin S, Wright SJ, eds. Optimization for Machine Learning (MIT Press, Cambridge, MA), 121–148.Google Scholar
• Juditsky A, Nemirovski A (2011b) First-order methods for nonsmooth convex large-scale optimization, II: Utilizing problem’s structure. Sra S, Nowozin S, Wright SJ, eds. Optimization for Machine Learning (MIT Press, Cambridge, MA), 149–184.Google Scholar
• Karaesmen IZ, Scheller-Wolf A, Deniz B (2011) Managing perishable and aging inventories: Review and future research directions. Kempf KG, Keskinocak P, Uzsoy R, eds. Planning Production and Inventories in the Extended Enterprise (Springer, New York), 393–436.Crossref, Google Scholar
• Klabjan D, Adelman D (2007) An infinite-dimensional linear programming algorithm for deterministic semi-Markov decision processes on Borel spaces. Math. Oper. Res. 32(3):528–550.Link, Google Scholar
• Nadarajah S, Margot F, Secomandi N (2015) Relaxations of approximate linear programs for the real option management of commodity storage. Management Sci. 61(12):3054–3076.Link, Google Scholar
• Nadarajah S, Secomandi N (2017) Relationship between least squares Monte Carlo and approximate linear programming. Oper. Res. Lett. 45(5):409–414.Crossref, Google Scholar
• Nahmias S, Smith SA (1994) Optimizing inventory levels in a two-echelon retailer system with partial lost sales. Management Sci. 40(5):582–596.Link, Google Scholar
• Nemirovski A, Juditsky A, Lan G, Shapiro A (2009) Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4):1574–1609.Crossref, Google Scholar
• Patrick J, Puterman M, Queyranne M (2008) Dynamic multipriority patient scheduling for a diagnostic resource. Oper. Res. 56(6):1507–1525.Link, Google Scholar
• Petrik M, Taylor G, Parr R, Zilberstein S (2010) Feature selection using regularization in approximate linear program for Markov decision processes. Fürnkranz J, Joachims T, eds. Proc. 27th Internat. Conf. Machine Learn. (Omnipress, Madison, WI), 871–878.Google Scholar
• Powell WB (2011) Approximate Dynamic Programming: Solving the Curses of Dimensionality, 2nd ed. (John Wiley & Sons, Hoboken, NJ).Crossref, Google Scholar
• Puterman ML (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, Inc., New York).Crossref, Google Scholar
• Rabinowitz G, Mehrez A, Chu C, Patuwo BE (1995) A partial backorder control for continuous review (r, q) inventory system with Poisson demand and constant lead time. Comput. Oper. Res. 22(7):689–700.Crossref, Google Scholar
• Restrepo M (2008) Computational methods for static allocation and real-time redeployment of ambulances. Unpublished doctoral dissertation, Cornell University, Ithaca, NY.Google Scholar
• Robert CP, Casella G (2004) Monte Carlo Statistical Methods (Springer, New York).Crossref, Google Scholar
• Schweitzer PJ, Seidmann A (1985) Generalized polynomial approximations in Markovian decision processes. J. Math. Anal. Appl. 110(2):568–582.Crossref, Google Scholar
• Secomandi N, Seppi DJ (2014) Real options and merchant operations of energy and other commodities. Foundations Trends Technol., Inform. Oper. Management 6(3–4):161–331.Crossref, Google Scholar
• Shapiro A, Dentcheva D, Ruszczyński A (2009) Lectures on Stochastic Programming: Modeling and Theory (SIAM, Philadelphia).Crossref, Google Scholar
• Sun P, Wang K, Zipkin P (2016) Quadratic approximation of cost functions in lost sales and perishable inventory control problems. Working paper, Duke University, Durham, NC.Google Scholar
• TensorFlow (2018) TensorFlow API documentation. Technical report. Accessed February 10, 2018, https://www.tensorflow.org/api_docs/python/tf/contrib/training/HParams.Google Scholar
• Tibshirani R (1996) Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B Methodology 58(1):267–288.Google Scholar
• Topaloglu H, Kunnumkal S (2006) Approximate dynamic programming methods for an inventory allocation problem under uncertainty. Naval Res. Logist. 53(8):822–841.Crossref, Google Scholar
• Trick MA, Zin SE (1997) Spline approximations to value functions. Macroeconom. Dynam. 1(1):255–277.Crossref, Google Scholar
• van de Ven PM, Hegde N, Massoulié L, Salonidis T (2013) Optimal control of end-user energy storage. IEEE Trans. Smart Grid 4(2):789–797.Crossref, Google Scholar
• Vossen TW, Zhang D (2015) Reductions of approximate linear programs for network revenue management. Oper. Res. 63(6):1352–1371.Link, Google Scholar
• Wang K (2014) Heuristics for inventory systems based on quadratic approximation of l#-convex value functions. Unpublished doctorial dissertation, Duke University, Durham, NC.Google Scholar
• Zhang D, Adelman D (2009) An approximate dynamic programming approach to network revenue management with customer choice. Transportation Sci. 43(3):381–394.Link, Google Scholar
• Zipkin P (2008a) Old and new methods for lost-sales inventory systems. Oper. Res. 56(5):1256–1263.Link, Google Scholar
• Zipkin P (2008b) On the structure of lost-sales inventory models. Oper. Res. 56(4):937–944.Link, Google Scholar