Latent Stratification for Incrementality Experiments

References

• AdExchanger (2021) Not just pass-fail: Why incrementality tests are the future of performance measurement. (February 23), https://www.adexchanger.com/data-driven-thinking/not-just-pass-fail-why-incrementality-tests-are-the-future-of-performance-measurement/.Google Scholar
• Athey S, Chetty R, Imbens GW, Kang H (2019) The surrogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. NBER Working Paper No. 26463, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Azevedo EM, Deng A, Olea JLM, Rao J, Weyl EG (2020) A/B testing with fat tails. J. Political Econom. 128(12):4614–4672.Crossref, Google Scholar
• Berman R, Van den Bulte C (2022) False discovery in A/B testing. Management Sci. 68(9):6762–6782.Link, Google Scholar
• Bonfrer A, Drèze X (2009) Real-time evaluation of email campaign performance. Marketing Sci. 28(2):251–263.Link, Google Scholar
• Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining KDD ‘16 (ACM, New York), 785–794.Google Scholar
• Deng A, Xu Y, Kohavi R, Walker T (2013) Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. Proc. Sixth ACM Internat. Conf. Web Search Data Mining (ACM, New York), 123–132.Google Scholar
• Ding P, Geng Z, Yan W, Zhou X-H (2011) Identifiability and estimation of causal effects by principal stratification with outcomes truncated by death. J. Amer. Statist. Assoc. 106(496):1578–1591.Crossref, Google Scholar
• Feit EM, Berman R (2019) Test & roll: Profit-maximizing A/B tests. Marketing Sci. 38(6):1038–1058.Link, Google Scholar
• Frangakis CE, Rubin DB (2002) Principal stratification in causal inference. Biometrics 58(1):21–29.Crossref, Google Scholar
• Gordon BR, Moakler R, Zettelmeyer F (2023) Close enough? A large-scale exploration of non-experimental approaches to advertising measurement. Marketing Sci. 42(4):768–793.Link, Google Scholar
• Gordon BR, Zettelmeyer F, Bhargava N, Chapsky D (2019) A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook. Marketing Sci. 38(2):193–225.Link, Google Scholar
• Guo Y, Coey D, Konutgan M, Li W, Schoener C, Goldman M (2021) Machine learning for variance reduction in online experiments. Adv. Neural Inform. Processing Systems 34:8637–8648.Google Scholar
• Hitsch GJ, Misra S (2018) Heterogeneous treatment effects and optimal targeting policy evaluation. Preprint, submitted February 6, https://dx.doi.org/10.2139/ssrn.3111957.Google Scholar
• Ho N, Feller A, Greif E, Miratrix L, Pillai N (2022) Weak separation in mixture models and implications for principal stratification. Internat. Conf. Artificial Intelligence Statist. (PMLR, New York), 5416–5458.Google Scholar
• Hoban PR, Bucklin RE (2015) Effects of internet display advertising in the purchase funnel: Model-based insights from a randomized field experiment. J. Marketing Res. 52(3):375–393.Crossref, Google Scholar
• Imbens GW, Rubin DB (2015) Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Johnson GA, Lewis RA, Nubbemeyer EI (2017a) Ghost ads: Improving the economics of measuring online ad effectiveness. J. Marketing Res. 54(6):867–884.Crossref, Google Scholar
• Johnson GA, Lewis RA, Reiley DH (2017b) When less is more: Data and power in advertising experiments. Marketing Sci. 36(1):43–53.Link, Google Scholar
• Kosyakova T, Pachali MJ, Smith AN, Otter T (2023) Constrained heterogeneity. Preprint, submitted January 23, https://dx.doi.org/10.2139/ssrn.4331470.Google Scholar
• Lewis RA, Rao JM (2015) The unfavorable economics of measuring the returns to advertising. Quart. J. Econom. 130(4):1941–1973.Crossref, Google Scholar
• Lewis RA, Wong J (2022) Incrementality bidding & attribution. Preprint, submitted August 25, https://arxiv.org/abs/2208.12809.Google Scholar
• Lin W (2013) Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. Ann. Appl. Statist. 7(1):295–318.Crossref, Google Scholar
• Miratrix LW, Sekhon JS, Yu B (2013) Adjusting treatment effect estimates by post-stratification in randomized experiments. J. Roy. Statist. Soc. Ser. B. Statist. Methodology 75(2):369–396.Crossref, Google Scholar
• Presnell B, Boos DD (2004) The IOS test for model misspecification. J. Amer. Statist. Assoc. 99(465):216–227.Crossref, Google Scholar
• Sahni NS (2016) Advertising spillovers: Evidence from online field experiments and implications for returns on advertising. J. Marketing Res. 53(4):459–478.Crossref, Google Scholar
• Sahni NS, Zou D, Chintagunta PK (2016) Do targeted discount offers serve as advertising? Evidence from 70 field experiments. Management Sci. 63(8):2688–2705.Link, Google Scholar
• Simester D, Timoshenko A, Zoumpoulis SI (2020) Efficiently evaluating targeting policies: Improving on champion vs. challenger experiments. Management Sci. 66(8):3412–3424.Link, Google Scholar
• Simester D, Timoshenko A, Zoumpoulis SI (2022) A sample size calculation for training and certifying targeting policies. Preprint, submitted October 2, https://dx.doi.org/10.2139/ssrn.4228297.Google Scholar
• Smith AN, Seiler S, Aggarwal I (2023) Optimal price targeting. Marketing Sci. 42(3):476–499.Link, Google Scholar
• Tibshirani J, Athey S, Friedberg R, Hadad V, Hirshberg D, Miner L, Sverdrup E, Wager S, Wright M (2018) Package ‘grf.’ Accessed August 22, 2023, https://rdrr.io/cran/grf/.Google Scholar
• Wager S, Athey S (2018) Estimation and inference of heterogeneous treatment effects using random forests. J. Amer. Statist. Assoc. 113(523):1228–1242.Crossref, Google Scholar
• Yang S (2021) Online experiments tricks—variance reduction. https://towardsdatascience.com/online-experiments-tricks-variance-reduction-291b6032dcd7.Google Scholar
• Yang J, Eckles D, Dhillon P, Aral S (2023) Targeting for long-term outcomes. Management Sci., ePub ahead of print August 3, https://doi.org/10.1287/mnsc.2023.4881.Google Scholar
• Yoganarasimhan H, Barzegary E, Pani A (2023) Design and evaluation of optimal free trials. Management Sci. 69(6):3220–3240.Link, Google Scholar
• Zantedeschi D, Feit EM, Bradlow ET (2016) Measuring multichannel advertising response. Management Sci. 63(8):2706–2728.Link, Google Scholar