Optimal Control of Partially Observable Semi-Markovian Failing Systems: An Analysis Using a Phase Methodology

References

• Altiok T (1985) On the phase-type approximations of general distributions. IIE Trans. 17(2):110–116.Crossref, Google Scholar
• Aven T , Bergman B (1986) Optimal replacement times: A general set-up. J. Appl. Probabilities 23(2):432–442.Crossref, Google Scholar
• Banjevic D , Kim MJ (2019) Thompson sampling for stochastic control: The continuous parameter case. IEEE Trans. Automated Control 64(10):4137–4152.Crossref, Google Scholar
• Barlow ER , Hunter L (1960) Optimum preventive maintenance policies. Oper. Res. 8(1):90–100.Link, Google Scholar
• Barlow ER , Proschan F (1975) Statistical Theory of Reliability and Life Testing: Probability Models (Holt, Rinehart and Winston, Inc., New York). Google Scholar
• Bertsekas DP , Shreve SE (1978) Stochastic Optimal Control: The Discrete Time Case (Academic Press, London).Google Scholar
• Boutros T , Liang M (2011) Detection and diagnosis of bearing and cutting tool faults using Hidden Markov models. Mech. Systems Signal Processing 25(6):2102–2124.Crossref, Google Scholar
• Cai J , Li H (2005) Multivariate risk model of phase type. Insurance Math. Econom. 36(2):137–152.Crossref, Google Scholar
• Derman C (1963) On Optimal Replacement Rules When Changes of State Are Markovian. Mathematical Optimization Techniques (University of California Press, Berkeley).Google Scholar
• Durante LJ , Nascimento J , Powell WB (2020) Backward approximate dynamic programming with hidden semi-markov stochastic models in energy storage optimization. Accessed October 11, 2017, https://arxiv.org/abs/1710.03914.Google Scholar
• Elbhbah K , Sinha JK (2013) Vibration-based condition monitoring of rotating machines using a machine composite spectrum. J. Sound Vibrations 332(11):2831–2845.Crossref, Google Scholar
• Elwany AH , Gebraeel NZ , Maillart LM (2011) Structured replacement policies for components with complex degradation processes and dedicated sensors. Oper. Res. 59(3):684–695.Link, Google Scholar
• Ferguson JD (1980) Variable duration models for speech. Proc. Sympos. on the Application of Hidden Markov Models to Text and Speech. 143–179.Google Scholar
• Grimmett GR , Stirzaker DR (2006) Probability and Random Processes (Oxford University Press, New York).Google Scholar
• Harper PR , Knight VA , Marshall AH (2012) Discrete conditional phase-type models utilizing classification trees: Application to modeling health service capacities. Eur. J. Oper. Res. 219(3):522–530.Crossref, Google Scholar
• Hernandez-Lerma O (1989) Adaptive Markov Control Processes (Springer, New York).Crossref, Google Scholar
• Jardine A , Lin D , Banjevic D (2006) A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mech. Systems Signal Processing 20(7):1483–1510.Crossref, Google Scholar
• Jardine AKS , Tsang AHC (2017) Maintenance, Replacement, and Reliability: Theory and Applications , 2nd ed. (Taylor and Francis Group, New York).Google Scholar
• Jardine AKS , Joseph T , Banjevic D (1999) Optimizing condition-based maintenance decisions for equipment subject to vibration monitoring. J. Qualitative Maintenance Engrg. 5(3):192–202.Crossref, Google Scholar
• Johnson MA , Taaffe MR (1989) Matching moments to phase distributions: Mixtures of Erlang distributions of common order. Stochastic Models 5(4):711–743.Crossref, Google Scholar
• Karlin S , Rinott Y (1980) Classes of orderings of measures and related correlation inequalities.I.multivariate totally positive distributions. J. Multivariate Anal. 10(4):467–498.Crossref, Google Scholar
• Khaleghei A , Makis V (2015) Model parameter estimation and residual life prediction for a partially observable failing system. Naval Res. Logist. 62(3):190–205.Crossref, Google Scholar
• Kharoufeh JP , Solo CJ , Ulukus MY (2010) Semi-Markov models for degradation-based reliability. IIE Trans. 42(8):599–612.Crossref, Google Scholar
• Kim MJ (2016) Robust control of partially observable failing systems. Oper. Res. 64(4):999–1014.Link, Google Scholar
• Kim MJ (2017) Thompson sampling for stochastic control: The finite parameter case. IEEE Trans. Automated Control 62(12):6415–6422.Crossref, Google Scholar
• Kim MJ (2020) Variance regularization in sequential Bayesian optimization. Math. Oper. Res. 45(3):966–992.Link, Google Scholar
• Kim MJ , Makis V (2013) Joint optimization of sampling and control of partially observable failing systems. Oper. Res. 61(3):777–790.Link, Google Scholar
• Kim MJ , Jiang R , Makis V , Lee CH (2011) Optimal Bayesian fault prediction scheme for a partially observable system subject to random failure. Eur. J. Oper. Res. 214(2):331–339.Crossref, Google Scholar
• Kim MJ , Makis V , Jiang R (2013) Parameter estimation for partially observable systems subject to random failure. Appl. Stochastic Models Bus. Industry 29(3):279–294.Crossref, Google Scholar
• Krishnamurthy V (2011) Bayesian sequential detection with phase-distributed change time and nonlinear penalty-a POMDP Lattice programming approach. IEEE Trans. Inform. Theory 57(10):7096–7124.Crossref, Google Scholar
• Kyung JY , Shaler Stidham JR (1983) Optimal service-rate control of M/G/1 queuing systems using phase methods. Adv. Appl. Probabilities 15(3):616–637.Crossref, Google Scholar
• Langen HJ (1982) Applying the method of phases in the optimization of queuing systems. Adv. Appl. Probabilities 14(1):122–142.Crossref, Google Scholar
• Liu Q , Dong M , Peng Y (2012) A novel method for online health prognosis of equipment based on hidden semi-Markov model using sequential Monte Carlo methods. Mech. Systems Signal Processing 32:331–348.Crossref, Google Scholar
• Lovejoy WS (1987) Some monotonicity results for partially observed Markov decision processes. Oper. Res. 35(5):736–743.Link, Google Scholar
• Maillart LM (2006) Maintenance policies for systems with condition monitoring and obvious failures. IIE Trans. 38(6):463–475.Crossref, Google Scholar
• Makis V (2008) Multivariate Bayesian control chart. Oper. Res. 56(2):487–496.Link, Google Scholar
• McClean S , Qiao Y , Fullerton K (2015) Stopgap: Stroke patient management and capacity planning. Oper. Res. Health Care 6:78–86.Crossref, Google Scholar
• Montoro-Cazorla D , Perez-Ocon R (2006) Reliability of a system under two types of failures using a Markovian arrival process. Oper. Res. Lett. 34(5):525–530.Crossref, Google Scholar
• Moustafaa M , Maksoudb EYA , Sadek S (2004) Optimal major and minimal maintenance policies for deteriorating systems. Reliability Engrg. Systems Safety 83(0):363–368.Crossref, Google Scholar
• Neutes MF , Perez-Ocon R , Torres-Castro I (2000) Repairable models with operating and repair times governed by phase type distributions. Adv. Appl. Probabilities 32(2):468–479.Crossref, Google Scholar
• Neuts MF (1981) Matrix-geometric solutions in stochastic models: An algorithmic approach (Johns Hopkins University Press, Baltimore).Google Scholar
• Perez-Ocon R , Ciencias FD (2004) Transient analysis of a repairable system, using phase-type distributions and geometric processes. IEEE Trans. Reliability 53(2):185–192.Crossref, Google Scholar
• Rinott Y , Scarsini M (2006) Total positivity order and the normal distribution. J. Multivariate Anal. 97:1251–1261.Crossref, Google Scholar
• Ross SM (2003) Introduction to Probability Models (Academic Press, San Diego).Google Scholar
• Schassberger R (1973) Warteschlangen (Springer-Verlag, New York).Crossref, Google Scholar
• Si XS , Wang W , Hu CH , Zhou DH (2011) Remaining useful life estimation: A review on the statistical data driven approaches. Eur. J. Oper. Res. 213(1):1–14.Crossref, Google Scholar
• Singh C , Billinton R , Lee S (1977) The method of stages for non-Markovian models. IEEE Trans. Reliability 26(1):135–137.Crossref, Google Scholar
• Smallwood RD , Sondik EJ (1973) The optimal control of partially observable Markov processes over a finite horizon. Oper. Res. 21(5):1071–1088.Link, Google Scholar
• Tijms HC (1990) Stochastic Modeling and Analysis: A Computational Approach (John Wiley and Sons, New York).Google Scholar
• Wang J , Lee CG (2015) Multi-state Bayesian control chart over a finite horizon. Oper. Res. 63(4):949–964.Link, Google Scholar
• White CC (1977) A Markov quality control process subject to partial observation. Management Sci. 23(8):843–852.Link, Google Scholar
• Whitt W (1979) A note on the influence of the sample on the posterior distribution. J. Amer. Statist. Assoc. 74(366):424–426.Crossref, Google Scholar
• Wiens DP (1979) Distributions for a two-unit repairable system. PhD. thesis, University of Calgary, Calgary, Alberta, Canada.Google Scholar
• Yang J , Makis V (2000) Dynamic response of residual to external deviations in a controlled production process. Technometrics 42(3):290–299.Crossref, Google Scholar
• Yang M , Makis V (2010) ARX model-based gearbox fault detection and localization under varying load conditions. J. Sound Vibrations 329(24):5209–5221.Crossref, Google Scholar
• Yu S (2010) Hidden semi-Markov models. Artificial Intelligence 174(2):215–243.Crossref, Google Scholar
• Zadeh AH , Bilodeau M (2013) Fitting bivariate losses with phase-type distributions. Scandanavian Actuarial J. (4):241–262.Crossref, Google Scholar
• Zhou Z , Hu C , Xu D , Chen M , Zhou D (2010) A model for real-time failure prognosis based on hidden Markov model and belief rule base. Eur. J. Oper. Res. 207(1):269–283.Crossref, Google Scholar