Policy Optimization in Dynamic Bayesian Network Hybrid Models of Biomanufacturing Processes

References

• Bankar AV, Kumar AR, Zinjarde SS (2009) Environmental and industrial applications of Yarrowia lipolytica. Appl. Microbiology Biotech. 84(5):847–865.Crossref, Google Scholar
• Bertsekas DP, Tsitsiklis JN (2000) Gradient convergence in gradient methods with errors. SIAM J. Optim. 10(3):627–642.Crossref, Google Scholar
• Cintron R (2015) Human factors analysis and classification system interrater reliability for biopharmaceutical manufacturing investigations. PhD thesis, Walden University, Minneapolis.Google Scholar
• Craven S, Whelan J, Glennon B (2014) Glucose concentration control of a fed-batch mammalian cell bioprocess using a nonlinear model predictive controller. J. Process Control 24(4):344–357.Crossref, Google Scholar
• Doran PM (2013) Bioprocess Engineering Principles (Academic Press, London).Google Scholar
• FDA (2009) Q8 pharmaceutical development. Technical report, U.S. Food & Drug Administration, Silver Spring, MD.Google Scholar
• Gelfand AE (2000) Gibbs sampling. J. Amer. Statist. Assoc. 95(452):1300–1304.Crossref, Google Scholar
• Ghadimi S, Lan G (2013) Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM J. Optim. 23(4):2341–2368.Crossref, Google Scholar
• Goh ATC (1995) Back-propagation neural networks for modeling complex systems. Artifical Intelligence Engrg. 9(3):143–151.Crossref, Google Scholar
• Gunther JC, Conner JS, Seborg DE (2009) Process monitoring and quality variable prediction utilizing PLS in industrial fed-batch cell culture. J. Process Control 19(5):914–921.Crossref, Google Scholar
• Hong MS, Severson KA, Jiang M, Lu AE, Love JC, Braatz RD (2018) Challenges and opportunities in biopharmaceutical manufacturing control. Comput. Chemical Engrg. 110:106–114.Crossref, Google Scholar
• Jain P, Kar P (2017) Non-convex optimization for machine learning. Foundations Trends Machine Learn. 10(3-4):142–363.Crossref, Google Scholar
• Jiang M, Braatz R (2016) Integrated control of continuous (bio)pharmaceutical manufacturing. Amer. Pharmaceutical Rev. 19(6):110–115.Google Scholar
• Kushner H, Yin G (2003) Stochastic Approximation and Recursive Algorithms and Applications, vol. 35, 2nd ed. (Springer, New York).Google Scholar
• Lakerveld R, Benyahia B, Braatz RD, Barton PI (2013) Model-based design of a plant-wide control strategy for a continuous pharmaceutical plant. AIChE J. 59(10):3671–3685.Crossref, Google Scholar
• Li X, Orabona F (2019) On the convergence of stochastic gradient descent with adaptive stepsizes. Chaudhuri K, Sugiyama M, eds. Proc. 22nd Internat. Conf. on Artificial Intelligence and Statist. (MIT Press, Cambridge, MA), 983–992.Google Scholar
• Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, et al. (2016) Continuous control with deep reinforcement learning. Bengio Y, LeCun Y, eds. Proc. 4th Internat. Conf. on Learn. Representations (ICLR, La Jolla, CA).Google Scholar
• Liu C, Gong Z, Shen B, Feng E (2013) Modelling and optimal control for a fed-batch fermentation process. Appl. Math. Modeling 37(3):695–706.Crossref, Google Scholar
• Lloyd I (2019) Pharma R&D annual review 2019. Technical report, Pharma Intelligence, London, UK.Google Scholar
• Lu AE, Paulson JA, Mozdzierz NJ, Stockdale A, Versypt ANF, Love KR, Love JC, et al. (2015) Control systems technology in the advanced manufacturing of biologic drugs. Proc. IEEE Conf. on Control Appl. (IEEE, Piscataway, NJ), 1505–1515.Google Scholar
• Mandenius CF, Titchener-Hooker NJ, Sonnleitner B, Stanke M, Hitzmann B, Chhatre S, Lencastre Fernandes R, Glassey J, Rathore AS (2013) Measurement, Monitoring, Modelling and Control of Bioprocesses, vol. 132 (Springer, Berlin).Crossref, Google Scholar
• Martagan T, Krishnamurthy A, Maravelias CT (2016) Optimal condition-based harvesting policies for biomanufacturing operations with failure risks. IIE Trans. 48(5):440–461.Crossref, Google Scholar
• Martagan T, Krishnamurthy A, Leland PA, Maravelias CT (2018) Performance guarantees and optimal purification decisions for engineered proteins. Oper. Res. 66(1):18–41.Link, Google Scholar
• Martin DK, Vicente O, Beccari T, Kellermayer M, Koller M, Lal R, Marks RS, et al. (2021) A brief overview of global biotechnology. Biotech. Biotech. Equipment 35(Suppl 1):S5–S14.Crossref, Google Scholar
• Matheron G, Perrin N, Sigaud O (2019) The problem with ddpg: Understanding failures in deterministic environments with sparse rewards. Preprint, submitted November 26, https://arxiv.org/abs/1911.11679.Google Scholar
• Nemirovski A, Juditsky A, Lan G, Shapiro A (2009) Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4):1574–1609.Crossref, Google Scholar
• Nesterov Y (2003) Introductory Lectures on Convex Optimization: A Basic Course (Springer, Boston, MA).Google Scholar
• O’Brien CM, Zhang Q, Daoutidis P, Hu WS (2021) A hybrid mechanistic-empirical model for in silico mammalian cell bioprocess simulation. Metabolism Engrg. 66:31–40.Crossref, Google Scholar
• Pandian BJ, Noel MM (2018) Control of a bioreactor using a new partially supervised reinforcement learning algorithm. J. Process Control 69:16–29.Crossref, Google Scholar
• Petsagkourakis P, Sandoval IO, Bradford E, Zhang D, del Rio-Chanona EA (2020) Reinforcement learning for batch bioprocess optimization. Comput. Chemical Engrg. 133:106649.Crossref, Google Scholar
• Powell WB (2010) Merging AI and or to solve high-dimensional stochastic optimization problems using approximate dynamic programming. INFORMS J. Comput. 22(1):2–17.Link, Google Scholar
• Powell WB (2011) Approximate Dynamic Programming: Solving the Curses of Dimensionality, 2nd ed. (John Wiley and Sons, New York).Crossref, Google Scholar
• Rathore AS, Bhushan N, Hadpe S (2011) Chemometrics applications in biotech processes: A review. Biotech. Progress 27(2):307–315.Crossref, Google Scholar
• Recht B, Re C, Wright S, Niu F (2011) Hogwild!: A lock-free approach to parallelizing stochastic gradient descent. Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira F, Weinberger KQ, eds. Adv. Neural Inform. Processings Systems (Curran Associates Inc., Red Hook, NY), 693–701.Google Scholar
• Shalev-Shwartz S, Singer Y, Srebro N, Cotter A (2011) Pegasos: Primal estimated sub-gradient solver for SVM. Math. Programming 127(1):3–30.Crossref, Google Scholar
• Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, et al. (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489.Crossref, Google Scholar
• Spielberg SPK, Gopaluni RB, Loewen PD (2017) Deep reinforcement learning approaches for process control. Proc. 6th Internat. Sympos. on Advanced Control of Industrial Processes (IEEE, Piscataway, NJ), 201–206.Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Teixeira AP, Carinhas N, Dias JML, Cruz P, Alves PM, Carrondo MJT, Oliveira R (2007) Hybrid semi-parametric mathematical systems: Bridging the gap between systems biology and process engineering. J. Biotech. 132(4):418–425.Crossref, Google Scholar
• Treloar NJ, Fedorec AJ, Ingalls B, Barnes CP (2020) Deep reinforcement learning for the control of microbial co-cultures in bioreactors. PLOS Comput. Biol. 16(4):e1007783.Crossref, Google Scholar
• Tsopanoglou A, del Val IJ (2021) Moving toward an era of hybrid modelling: Advantages and challenges of coupling mechanistic and data-driven models for upstream pharmaceutical bioprocesses. Curr. Opinion Chemical Engrg. 32:100691.Crossref, Google Scholar
• Varga E, Titchener-Hooker N, Dunnill P (2001) Prediction of the pilot-scale recovery of a recombinant yeast enzyme using integrated models. Biotech. Bioengrg. 74(2):96–107.Crossref, Google Scholar
• Xie W, Wang B, Li C, Xie D, Auclair J (2020) Interpretable biomanufacturing process risk and sensitivity analyses for quality-by-design and stability control. Naval Res. Logist. 69(3):461–483.Google Scholar
• Zhang D, Del Rio-Chanona EA, Petsagkourakis P, Wagner J (2019) Hybrid physics-based and data-driven modeling for bioprocess online simulation and optimization. Biotech. Bioengrg. 116(11):2919–2930.Crossref, Google Scholar
• Zheng H, Ryzhov IO, Xie W, Zhong J (2021) Personalized multimorbidity management for patients with type 2 diabetes using reinforcement learning of electronic health records. Drugs 81(4):471–482.Crossref, Google Scholar