A Regularized High-Dimensional Positive Definite Covariance Estimator with High-Frequency Data

Liyuan Cui
Liyuan Cui
[email protected]
https://orcid.org/0000-0002-7236-0684
Department of Economics and Finance, City University of Hong Kong, Hong Kong SAR;
Search for more papers by this author
,
Yongmiao Hong
Corresponding Author
Yongmiao Hong
[email protected]
https://orcid.org/0000-0002-9711-9061
Center for Forecasting Science, Chinese Academy of Sciences, Beijing 100045, China;School of Economics and Management, and MOE Social Science Laboratory of Digital Economic Forecasts and Policy Simulation, University of Chinese Academy of Sciences, Beijing 100190, China;Department of Economics, Cornell University, Ithaca, New York 14850;
Search for more papers by this author
,
Yingxing Li
Yingxing Li
[email protected]
https://orcid.org/0000-0002-1634-9253
Wang Yanan Institute of Studies in Economics, Xiamen University, Fujian 361005, China;
Search for more papers by this author
,
Junhui Wang
Junhui Wang
[email protected]
https://orcid.org/0000-0002-9165-5664
Department of Statistics, Chinese University of Hong Kong, Hong Kong SAR
Search for more papers by this author

Department of Economics and Finance, City University of Hong Kong, Hong Kong SAR;

Corresponding Author

Yongmiao Hong

Center for Forecasting Science, Chinese Academy of Sciences, Beijing 100045, China;School of Economics and Management, and MOE Social Science Laboratory of Digital Economic Forecasts and Policy Simulation, University of Chinese Academy of Sciences, Beijing 100190, China;Department of Economics, Cornell University, Ithaca, New York 14850;

Search for more papers by this author

Yingxing Li

[email protected]

https://orcid.org/0000-0002-1634-9253

Wang Yanan Institute of Studies in Economics, Xiamen University, Fujian 361005, China;

Search for more papers by this author

Junhui Wang

[email protected]

https://orcid.org/0000-0002-9165-5664

Department of Statistics, Chinese University of Hong Kong, Hong Kong SAR

Search for more papers by this author

Published Online:12 Dec 2023https://doi.org/10.1287/mnsc.2022.04138

References

Agarwal A, Negahban S, Wainwright MJ (2012) Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions. Ann. Statist. 40(2):1171–1197.Crossref, Google Scholar
Ahn SC, Horenstein AR (2013) Eigenvalue ratio test for the number of factors. Econometrica 81(3):1203–1227.Crossref, Google Scholar
Aït-Sahalia Y, Xiu D (2017) Using principal component analysis to estimate a high dimensional factor model with high-frequency data. J. Econometrics 201(2):384–399.Crossref, Google Scholar
Aït-Sahalia Y, Xiu D (2019) Principal component analysis of high-frequency data. J. Amer. Statist. Assoc. 114(525):287–303.Crossref, Google Scholar
Aït-Sahalia Y, Fan J, Xiu D (2010) High-frequency covariance estimates with noisy and asynchronous financial data. J. Amer. Statist. Assoc. 105(492):1504–1517.Crossref, Google Scholar
Aït-Sahalia Y, Kalnina I, Xiu D (2020) High-frequency factor models and regressions. J. Econometrics 216(1):86–105.Crossref, Google Scholar
Anatolyev S, Mikusheva A (2022) Factor models with many assets: Strong factors, weak factors, and the two-pass procedure. J. Econometrics 229(1):103–126.Crossref, Google Scholar
Andersen TG, Fusari N, Todorov V, Varneskov RT (2019) Unified inference for nonlinear factor models from panels with fixed and large time span. J. Econometrics 212(1):4–25.Crossref, Google Scholar
Bai J, Li K (2012) Statistical analysis of factor models of high dimension. Ann. Statist. 40(1):436–465.Crossref, Google Scholar
Bai J, Ng S (2002) Determining the number of factors in approximate factor models. Econometrica 70(1):191–221.Crossref, Google Scholar
Bai J, Ng S (2006) Evaluating latent and observed factors in macroeconomics and finance. J. Econometrics 131(1–2):507–537.Crossref, Google Scholar
Bai J, Ng S (2007) Determining the number of primitive shocks in factor models. J. Bus. Econom. Statist. 25(1):52–60.Crossref, Google Scholar
Bai J, Ng S (2019) Rank regularized estimation of approximate factor models. J. Econometrics 212(1):78–96.Crossref, Google Scholar
Bai J, Ng S (2023) Approximate factor models with weaker loadings. J. Econometrics 235(2):1893–1916.Crossref, Google Scholar
Bailey N, Kapetanios G, Pesaran MH (2021) Measurement of factor strength: Theory and practice. J. Appl. Econometrics 36(5):587–613.Crossref, Google Scholar
Bandi FM, Russell JR (2008) Microstructure noise, realized variance, and optimal sampling. Rev. Econom. Stud. 75(2):339–369.Crossref, Google Scholar
Barndorff-Nielsen OE, Shephard N (2004) Power and bipower variation with stochastic volatility and jumps. J. Financial Econom. 2(1):1–37.Crossref, Google Scholar
Boudt K, Laurent S, Lunde A, Quaedvlieg R, Sauri O (2017) Positive semidefinite integrated covariance estimation, factorizations and asynchronicity. J. Econometrics 196(2):347–367.Crossref, Google Scholar
Chang J, Hu Q, Liu C, Tang CY (2022) Optimal covariance matrix estimation for high-dimensional noise in high-frequency data. J. Econometrics, ePub ahead of print September 16, https://doi.org/10.1016/j.jeconom.2022.06.010.Google Scholar
Christensen K, Kinnebrock S, Podolskij M (2010) Pre-averaging estimators of the ex-post covariance matrix in noisy diffusion models with non-synchronous data. J. Econometrics 159(1):116–133.Crossref, Google Scholar
Dai C, Lu K, Xiu D (2019) Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data. J. Econometrics 208(1):43–79.Crossref, Google Scholar
Duan J, Pelger M, Xiong R (2022) Target PCA: Transfer learning large dimensional panel data. Preprint, submitted December 27, https://dx.doi.org/10.2139/ssrn.4302869.Google Scholar
Efron B (2004) The estimation of prediction error: Covariance penalties and cross-validation. J. Amer. Statist. Assoc. 99(467):619–632.Crossref, Google Scholar
Fama EF, French KR (1993) Common risk factors in the returns on stocks and bonds. J. Financial Econom. 33(1):3–56.Crossref, Google Scholar
Fama EF, French KR (2015) A five-factor asset pricing model. J. Financial Econom. 116(1):1–22.Crossref, Google Scholar
Fan J, Kim D (2018) Robust high-dimensional volatility matrix estimation for high-frequency factor model. J. Amer. Statist. Assoc. 113(523):1268–1283.Crossref, Google Scholar
Fan J, Kim D (2019) Structured volatility matrix estimation for non-synchronized high-frequency financial data. J. Econometrics 209(1):61–78.Crossref, Google Scholar
Fan J, Furger A, Xiu D (2016) Incorporating global industrial classification standard into portfolio allocation: A simple factor-based large covariance matrix estimator with high-frequency data. J. Bus. Econom. Statist. 34(4):489–503.Crossref, Google Scholar
Fan J, Liao Y, Mincheva M (2013) Large covariance estimation by thresholding principal orthogonal complements. J. Royal Statist. Soc. Ser. B Statist. Methodology 75(4):603–680.Crossref, Google Scholar
Giglio S, Xiu D (2021) Asset pricing with omitted factors. J. Political Econom. 129(7):1947–1990.Crossref, Google Scholar
Hallin M, Liška R (2007) Determining the number of factors in the general dynamic factor model. J. Amer. Statist. Assoc. 102(478):603–617.Crossref, Google Scholar
Huang D, Jiang F, Li K, Tong G, Zhou G (2022) Scaled PCA: A new approach to dimension reduction. Management Sci. 68(3):1678–1695.Link, Google Scholar
Kim D, Wang Y, Zou J (2016) Asymptotic theory for large volatility matrix estimation based on high-frequency financial data. Stochastic Processing Appl. 126(11):3527–3577.Crossref, Google Scholar
Kim D, Kong XB, Li CX, Wang Y (2018) Adaptive thresholding for large volatility matrix estimation based on high-frequency financial data. J. Econometrics 203(1):69–79.Crossref, Google Scholar
Kong XB (2017) On the number of common factors with high-frequency data. Biometrika 104(2):397–410.Crossref, Google Scholar
Kong XB (2018) On the systematic and idiosyncratic volatility with large panel high-frequency data. Ann. Statist. 46(3):1077–1108.Crossref, Google Scholar
Kong XB, Lin JG, Liu C, Liu GY (2023) Discrepancy between global and local principal component analysis on large-panel high-frequency data. J. Amer. Statist. Assoc. 118(542):1333–1344.Crossref, Google Scholar
Lettau M, Pelger M (2020) Estimating latent asset-pricing factors. J. Econometrics 218(1):1–31.Crossref, Google Scholar
Onatski A (2010) Determining the number of factors from empirical distribution of eigenvalues. Rev. Econom. Statist. 92(4):1004–1016.Crossref, Google Scholar
Onatski A (2012) Asymptotics of the principal components estimator of large factor models with weakly influential factors. J. Econometrics 168(2):244–258.Crossref, Google Scholar
Pelger M (2019) Large-dimensional factor modeling based on high-frequency observations. J. Econometrics 208(1):23–42.Crossref, Google Scholar
Pelger M (2020) Understanding systematic risk: A high-frequency approach. J. Finance 75(4):2179–2220.Crossref, Google Scholar
Pelger M, Xiong R (2022) State-varying factor models of large dimensions. J. Bus. Econom. Statist. 40(3):1315–1333.Crossref, Google Scholar
Shin M, Kim D, Fan J (2021) Adaptive robust large volatility matrix estimation based on high-frequency financial data. Preprint, submitted March 26, https://dx.doi.org/10.2139/ssrn.3793394.Google Scholar
Stein CM (1981) Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9(6):1135–1151.Crossref, Google Scholar
Stock JH, Watson MW (2002) Forecasting using principal components from a large number of predictors. J. Amer. Statist. Assoc. 97(460):1167–1179.Crossref, Google Scholar
Tao M, Wang Y, Zhou HH (2013) Optimal sparse volatility matrix estimation for high-dimensional itô processes with measurement errors. Ann. Statist. 41(4):1816–1864.Crossref, Google Scholar
Wang Y, Zou J (2010) Vast volatility matrix estimation for high-frequency financial data. Ann. Statist. 38(2):943–978.Crossref, Google Scholar
Xue L, Ma S, Zou H (2012) Positive-definite ℓ1-penalized estimation of large covariance matrices. J. Amer. Statist. Assoc. 107(500):1480–1491.Crossref, Google Scholar
Ye J (1998) On measuring and correcting the effects of data mining and model selection. J. Amer. Statist. Assoc. 93(441):120–131.Crossref, Google Scholar

Volume 70, Issue 10

October 2024

Pages vii-xii, 6483-7343, iv-vi

Article Information

Supplemental Material

Metrics

Information

Received:September 15, 2021
Accepted:June 04, 2023
Published Online:December 12, 2023

Cite as

Liyuan Cui, Yongmiao Hong, , Yingxing Li, Junhui Wang (2023) A Regularized High-Dimensional Positive Definite Covariance Estimator with High-Frequency Data. Management Science 70(10):7242-7264.

https://doi.org/10.1287/mnsc.2022.04138

Keywords

Acknowledgments

The authors thank the department editor (Agostino Capponi), associate editor, and two anonymous referees for valuable suggestions. The authors are particularly grateful to the editor and associate editor for suggesting analyzing weak factors presented in Section 3 of the paper. The authors also thank Guanghao Feng, Jingyu He, Jiti Gao, Taehwy Lee, Cheng Liu, Whitney Newey, Aman Ullah, Xueping Wu, and Shuai Ye for helpful comments. Jingyi Yao, a PhD student at the City University of Hong Kong, provided excellent research assistance. The authors thank Zengguang Zhong and Jiajun Yuan, senior engineers from MOE Key Laboratory of Econometrics Xiamen University, for supplying parallel computing support.

PDF download

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

A Regularized High-Dimensional Positive Definite Covariance Estimator with High-Frequency Data

References

Volume 70, Issue 10

Article Information

Supplemental Material

Metrics

Information

Cite as

Keywords

Sign Up for INFORMS Publications Updates and News