Overall Evaluation Criteria for A/B Tests: A Meta-Analytic Approach
References
- (2003) E-customization. J. Marketing Res. 40(2):131–145.Crossref, Google Scholar
- (2018) Retention futility: Targeting high-risk customers might be ineffective. J. Marketing Res. 55(1):80–98.Crossref, Google Scholar
- (2026) The surrogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. Rev. Econom. Stud. 93(4):2284–2312.Crossref, Google Scholar
- (2019) Empirical Bayes estimation of treatment effects with many A/B tests: An overview. AEA Papers Proc. 109:43–47.Crossref, Google Scholar
- (2020) A/B testing with fat tails. J. Political Econom. 128(12):4614–4672.Crossref, Google Scholar
- (1985) Statistical Decision Theory and Bayesian Analysis, Springer Series in Statistics, 2nd ed. (Springer-Verlag, New York).Crossref, Google Scholar
- (2018) Beyond the last touch: Attribution in online advertising. Marketing Sci. 37(5):771–792.Link, Google Scholar
- (2006) Variational inference for Dirichlet process mixtures. Bayesian Anal. 1(1):121–143.Crossref, Google Scholar
- (2011) Handbook of Markov Chain Monte Carlo (CRC Press, Boca Raton, FL).Crossref, Google Scholar
- (1973) Measurable selections of extrema. Ann. Statist. 1(5):902–912.Crossref, Google Scholar
- (2020) Scalable target marketing: Distributed Markov chain Monte Carlo for Bayesian hierarchical models. J. Marketing Res. 57(6):999–1018.Crossref, Google Scholar
- (2005) A Bayesian semiparametric model for random-effects meta-analysis. J. Amer. Statist. Assoc. 100(469):242–251.Crossref, Google Scholar
- (2000) The validation of surrogate endpoints in meta-analyses of randomized experiments. Biostatistics 1(1):49–67.Crossref, Google Scholar
- (2017) Links between causal effects and causal association for surrogacy evaluation in a Gaussian setting. Statist. Medicine 36(27):4243–4265.Crossref, Google Scholar
- (1954) The multivariate t-distribution associated with a set of normal sample deviates. Australian J. Phys. 7(4):531–542.Crossref, Google Scholar
- (2015) Objective Bayesian two sample hypothesis testing for online controlled experiments. Gangemi A, Leonardi S, Panconesi A (General Chairs), eds. Proc. 24th Internat. Conf. World Wide Web (Association for Computing Machinery, New York), 923–928.Google Scholar
- (2016) Data-driven metric development for online controlled experiments: Seven lessons learned. Krishnapuram B, Shah M (General Chairs), Smola A, Aggarwal C, Shen D, Rastogi R (Program Chairs), eds. Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 77–86.Google Scholar
- (2013) Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. Leonardi S, Panconesi A (General Chairs), Ferragina P, Gionis A (Program Chairs), eds. Proc. Sixth ACM Internat. Conf. Web Search Data Mining (Association for Computing Machinery, New York), 123–132.Google Scholar
- (2023) From augmentation to decomposition: A new look at CUPED in 2023. Preprint, submitted December 5, https://doi.org/10.48550/arXiv.2312.02935.Google Scholar
- (1986) Meta-analysis in clinical trials. Controlled Clinical Trials 7(3):177–188.Crossref, Google Scholar
- (2015) Meta-analysis in clinical trials revisited. Contemporary Clinical Trials 45(Pt A):139–145.Crossref, Google Scholar
- (2018) Bayesian nonparametric customer base analysis with model-based visualizations. Marketing Sci. 37(2):216–235.Link, Google Scholar
- (2016) On the conditional distribution of the multivariate t distribution. Amer. Statistician 70(3):293–295.Crossref, Google Scholar
- (2014) Two modeling strategies for empirical Bayes estimation. Statist. Sci. 29(2):285–301.Crossref, Google Scholar
- (2023) Surrogate endpoints in clinical trials. Annual Rev. Statist. Appl. 10(1):75–96.Crossref, Google Scholar
- (2015) Surrogacy marker paradox measures in meta-analytic settings. Biostatistics 16(2):400–412.Crossref, Google Scholar
- (1995) Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90(430):577–588.Crossref, Google Scholar
- (2024) Causal machine learning for predicting treatment outcomes. Nature Medicine 30(4):958–968.Crossref, Google Scholar
- (1996) Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica 6(4):733–760.Google Scholar
- (2021) Data, measurement, and causal inferences in machine learning: Opportunities and challenges for marketing. J. Marketing Theory Practice 29(1):65–77.Crossref, Google Scholar
- (1950) Estimation of genetic parameters (abstract). Ann. Math. Statist. 21(2):309–310.Google Scholar
- (2009) A re-evaluation of random-effects meta-analysis. J. Roy Statist. Soc. Ser. A Statist. Soc. 172(1):137–159.Crossref, Google Scholar
- (2026) Estimating effects of long-term treatments. Management Sci., ePub ahead of print April 9, https://doi.org/10.1287/mnsc.2023.02575.Google Scholar
- (2024) Long-term causal inference under persistent confounding via data combination. J. Roy Statist. Soc. Ser. B Statist. Methodology 87(2):362–388.Crossref, Google Scholar
- (2024) Learning metrics that maximise power for accelerated A/B-tests. Baeza-Yates R, Bonchi F (General Chairs), eds. Proc. 30th ACM SIGKDD Conf. Knowledge Discovery Data Mining KDD 24 (Association for Computing Machinery, New York), 5183–5193.Google Scholar
- (2023) Toward optimal variance reduction in online controlled experiments. Technometrics 65(2):231–242.Crossref, Google Scholar
- (2009) Related causal frameworks for surrogate outcomes. Biometrics 65(2):530–538.Crossref, Google Scholar
- (2004) Assessing heterogeneity in discrete choice models using a Dirichlet process prior. Rev. Marketing Sci. 2(1):1–41.Crossref, Google Scholar
- (2014) Convex optimization, shape constraints, compound decisions, and empirical Bayes rules. J. Amer. Statist. Assoc. 109(506):674–685.Crossref, Google Scholar
- (2020) Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing, 1st ed. (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
- (2015) The unfavorable economics of measuring the returns to advertising. Quart. J. Econom. 130(4):1941–1973.Crossref, Google Scholar
- (1994) Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81(1):27–40.Crossref, Google Scholar
- (1994) Posterior predictive p-values. Ann. Statist. 22(3):1142–1160.Crossref, Google Scholar
- (2012) Algorithms for hierarchical clustering: An overview. WIREs Data Mining Knowledge Discovery 2(1):86–97.Crossref, Google Scholar
- (2000) Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Statist. 9(2):249–265.Crossref, Google Scholar
- (2011) MCMC using Hamiltonian dynamics. Brooks S, Gelman A, Jones GL, Meng XL, eds. Handbook of Markov Chain Monte Carlo (CRC Press, Boca Raton, FL), 113–162.Crossref, Google Scholar
- (2009) A survey on transfer learning. IEEE Trans. Knowledge Data Engrg. 22(10):1345–1359.Crossref, Google Scholar
- (2023) A matrix completion solution to the problem of ignoring the ignorability assumption. Preprint, submitted September 6, https://dx.doi.org/10.2139/ssrn.4546996.Google Scholar
- (2021) On the Bernstein–von Mises theorem for the Dirichlet process. Electronic J. Statist. 15(1):2224–2246.Crossref, Google Scholar
- (1991) That BLUP is a good thing: The estimation of random effects. Statist. Sci. 6(1):15–32.Crossref, Google Scholar
- (2013) Nonparametric Bayesian inference. NSF–CBMS Regional Conf. Ser. Probab. Statist., vol. 9, i–110.Google Scholar
- (2024) Bayesian Statistics and Marketing, Wiley Series in Probability and Statistics, 2nd ed. (John Wiley & Sons, Newark, NJ).Crossref, Google Scholar
- (2016) On the frequentist properties of Bayesian nonparametric methods. Annual Rev. Statist. Appl. 3(1):211–231.Crossref, Google Scholar
- (1974) Estimating causal effects of treatments in randomized and nonrandomized studies. J. Ed. Psych. 66(5):688–701.Crossref, Google Scholar
- (1984) Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12(4):1151–1172.Crossref, Google Scholar
- (2024) Some properties of GEM(α) distributions. Sankhya A 86(Suppl 1):288–300.Crossref, Google Scholar
- (2002) Bayesian measures of model complexity and fit. J. Roy Statist. Soc. Ser. B Statist. Methodology 64(4):583–639.Crossref, Google Scholar
- (2010) Mind-set metrics in market response models: An integrative approach. J. Marketing Res. 47(4):672–684.Crossref, Google Scholar
- (1987) The calculation of posterior distributions by data augmentation. J. Amer. Statist. Assoc. 82(398):528–540.Crossref, Google Scholar
- (2024) Choosing a proxy metric from past experiments. Baeza-Yates R, Bonchi F (General Chairs), eds. Proc. 30th ACM SIGKDD Conf. Knowledge Discovery Data Mining KDD 24 (Association for Computing Machinery, New York), 5803–5812.Google Scholar
- (2001) The art of data augmentation (with discussion). J. Comput. Graph. Statist. 10(1):1–50.Crossref, Google Scholar
- (2002) Advanced methods in meta-analysis: Multivariate approach and meta-regression. Statist. Medicine 21(4):589–624.Crossref, Google Scholar
- (2010) Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J. Machine Learn. Res. 11(116):3571–3594.Google Scholar
- (2024) Targeting for long-term outcomes. Management Sci. 70(6):3841–3855.Link, Google Scholar

