A Deep Q-Network for the Beer Game: Deep Reinforcement Learning for Inventory Optimization

References

• Ban G-Y, Rudin C (2019) The big data newsvendor: Practical insights from machine learning. Oper. Res. 67(1):90–108.Link, Google Scholar
• Ban G-Y, Gallien J, Mersereau AJ (2019) Dynamic procurement of new products with covariate information: The residual tree method. Manufacturing Service Oper. Management 21(4):798–815.Link, Google Scholar
• Bernstein DS, Givan R, Immerman N, Zilberstein S (2002) The complexity of decentralized control of Markov decision processes. Math. Oper. Res. 27(4):819–840.Link, Google Scholar
• Chaharsooghi SK, Heydari J, Zegordi SH (2008) A reinforcement learning model for supply chain ordering management: An application to the beer game. Decision Support Systems 45(4):949–959.Crossref, Google Scholar
• Chan D (2017) The AI that has nothing to learn from humans. Atlantic 20:354–359.Google Scholar
• Chen F, Samroengraja R (2000) The stationary beer game. Production Oper. Management 9(1):19.Crossref, Google Scholar
• Chen F, Zheng Y (1994) Lower bounds for multi-echelon stochastic inventory systems. Management Sci. 40(11):1426–1443.Link, Google Scholar
• Clark AJ, Scarf H (1960) Optimal policies for a multi-echelon inventory problem. Management Sci. 6(4):475–490.Link, Google Scholar
• Claus C, Boutilier C (1998) The dynamics of reinforcement learning in cooperative multiagent systems. The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems, (American Association for Artificial Intelligence, Madison, WI), 746–752.Google Scholar
• Croson R, Donohue K (2006) Behavioral causes of the bullwhip effect and the observed value of inventory information. Management Sci. 52(3):323–336.Link, Google Scholar
• Gallego G, Zipkin P (1999) Stock positioning and performance estimation in serial production-transportation systems. Manufacturing Service Oper. Management 1(1):77–88.Link, Google Scholar
• Geary S, Disney SM, Towill DR (2006) On bullwhip in supply chains—Historical review, present practice and expected future impact. Internat. J. Production Econom. 101(1):2–18.Crossref, Google Scholar
• Giannoccaro I, Pontrandolfo P (2002) Inventory management in supply chains: A reinforcement learning approach. Internat. J. Production Econom. 78(2):153–161.Google Scholar
• Gijsbrechts J, Boute RN, Van Mieghem JA, Zhang D (2019) Can deep reinforcement learning improve inventory management? Performance on dual sourcing, lost sales and multi-echelon problems. https://ssrn.com/abstract=3302881 or http://dx.doi.org/10.2139/ssrn.3302881.Google Scholar
• Graves SC (1985) A multi-echelon inventory model for a repairable item with one-for-one replenishment. Management Sci. 31(10):1247–1256.Link, Google Scholar
• Jiang C, Sheng Z (2009) Case-based reinforcement learning for dynamic inventory control in a multi-agent supply chain system. Expert Systems Appl. 36(3):6520–6526.Crossref, Google Scholar
• Kaggle.com (2018) Store item demand forecasting challenge. Accessed August 26, 2019, https://www.kaggle.com/c/demand-forecasting-kernels-only/overview.Google Scholar
• Kimbrough SO, Wu D-J, Zhong F (2002) Computers play the beer game: Can artificial agents manage supply chains? Decision Support Systems 33(3):323–333.Crossref, Google Scholar
• Lee HL, Padmanabhan V, Whang S (1997) Information distortion in a supply chain: The bullwhip effect. Management Sci. 43(4):546–558.Link, Google Scholar
• Lee HL, Padmanabhan V, Whang S (2004) Comments on “Information distortion in a supply chain: The bullwhip effect.” Management Sci. 50(12S):1887–1893.Link, Google Scholar
• Li Y (2017) Deep reinforcement learning: An overview. Preprint, submitted XX, https://arxiv.org/abs/1701.07274.Google Scholar
• Martinez-Moyano IJ, Rahn J, Spencer R (2014) The beer game: Its history and rule changes. Technical report, University at Albany, Albany, NY.Google Scholar
• Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, et al. (2016) Asynchronous methods for deep reinforcement learning. Balcan MF, Weinberger KQ, eds. Proc. 33rd Int. Conf. Mach. Learn. (PMLR, New York), Vol. 48, 1928–1937.Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Oroojlooyjadid A, Hajinezhad D (2019) A review of cooperative multi-agent deep reinforcement learning. Preprint, submitted XX, https://arxiv.org/abs/1908.03963.Google Scholar
• Oroojlooyjadid A, Snyder L, Takáč M (2017) Stock-out prediction in multi-echelon networks. Preprint, submitted XX, https://arxiv.org/abs/1709.06922.Google Scholar
• Oroojlooyjadid A, Snyder LV, Takáč M (2020) Applying Deep Learning to the Newsvendor Problem (Taylor & Francis), 444–463.Google Scholar
• Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans. Knowledge Data Engrg. 22(10):1345–1359.Crossref, Google Scholar
• Pentaho (2008) Foodmart’s database tables. Accessed September 30, 2015, http://pentaho.dlpage.phi-integration.com/mondrian/mysql-foodmart-database.Google Scholar
• Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, et al. (2017) Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. Preprint, submitted XX, https://arxiv.org/abs/1711.05225.Google Scholar
• Snyder LV (2018) Multi-echelon base-stock optimization with upstream stockout costs. Technical report, Lehigh University, Bethlehem, PA.Google Scholar
• Snyder LV, Shen Z-JM (2019) Fundamentals of Supply Chain Theory, 2nd ed. (John Wiley & Sons, Hoboken, NJ).Crossref, Google Scholar
• Sterman JD (1989) Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment. Management Sci. 35(3):321–339.Link, Google Scholar
• Strozzi F, Bosch J, Zaldivar J (2007) Beer game order policy optimization under changing customer demand. Decision Support Systems 42(4):2153–2163.Crossref, Google Scholar
• Sutton RS, Barto AG (1998) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Yang Z, Xie Y, Wang Z (2019) A theoretical analysis of deep Q-learning. Proc Machine Learning Res. 120:486–489, https://arxiv.org/abs/1901.00137.Google Scholar
• Zhang K, Yang Z, Başar T (2019) Multi-agent reinforcement learning: A selective overview of theories and algorithms. Preprint, submitted XX, https://arxiv.org/abs/1911.10635.Google Scholar